Reading strings of floats and saving in Teensy EEPROM

Status
Not open for further replies.

amensch

Well-known member
Hello All,
This code reads in a string of floating point numbers to the Teensy and places it it an array. What I am having trouble with is then storing the values in EEPROM, so they don;t have to be updated every time new firmware is pushed to the Teensy.
Internally, it appears a float is 32 bits on a Teensy 4.0, and I assume it's represented in IEEE754 format on the actual internal data bus. I'm trying a shortcut to this method,
https://github.com/MalcolmMcLean/ieee754/blob/master/binaryio.c which actually looks at the parts of the float. I'm probably mistaken, but it seems to me that one should be able to break the 32 bit float into bytes and write them to 4 locations in EEPROM, then, use the reverse process to re-create the original float.

But, I get this error
Code:
invalid operands of types 'float' and 'int' to binary 'operator>>'
when I try to make a byte conversion in here:
Code:
float fwriteToEEPROM(float floatToWrite, int fp){
        int n;
        unsigned char toEEPROM;
        for ( n=0; n<4; n++){
          toEEPROM = (char)(floatToWrite >> n*8) & 0xFF;
          EEPROM.write(toEEPROM, fp+n); delay(4);
        }
return freadFromEEPROM(int fp);  // Return what was written to confirm.
}
The complete sketch is attached, feel free to use this if helpful.
Regards All,
-jb
 

Attachments

  • T4_FloatStringsInput.ino
    5.8 KB · Views: 56
Arduino has this :: arduino.cc/en/Reference/EEPROMPut

With an associated Get() to read and write float or other types. It works efficiently to update the EEPROM area as needed with PJRC tested code - and the EEPROM read/write works on 32 bits IIRC Paul noted in past days.

You just need to remember to index by sizeof(type) and keep the eeprom addresses right.
 
I use these in a header file:

Code:
    // Note: DO NOT USE WITH CHAR[].
    template <class T> int read(int addr, T& value) {
        //addr *= 4;
        byte* p = (byte*)(void*)&value;
        unsigned int i;
        for (i = 0; i < sizeof(value); i++)
            *p++ = EEPROM.read(addr++);
        return i;
    }

    // Note: DO NOT USE WITH CHAR[].
    template <class T> int write(int addr, const T& value) {
        //addr *= 4;
        const byte* p = (const byte*)(const void*)&value;
        unsigned int i;
        for (i = 0; i < sizeof(value); i++) {
            EEPROM.write(addr++, *p++);
        }
        return i;
    }

The "addr *= 4" lines are commented out because not all data types are 4 bytes wide. This means you have to supply the correct byte address.
 
Thanks Pilot...... I will need to let the sophistication there sink into my caveman brain a while.
In the meantime, this is just merged from the EEPROM.get & EEPROM.put examples...
Code:
#include <EEPROM.h>
struct MyObject{  float field1;  byte field2;  char name[10];};
MyObject customVar = { 3.14f,    65,    "Working!"  };

void setup(){
  float f = 123456789.123456789f;   //Variable to store data read from EEPROM.
  int eeAddress = 100; //EEPROM address to start reading from

  Serial.begin( 115200 );
  while (!Serial) {   ;   }  // wait for serial port to connect. Needed for Leonardo only
  
  Serial.print( "Put float in EEPROM: " );
  EEPROM.put(eeAddress, f);  
  Serial.println("Written float data type!");    /** Put is designed for use with custom structures also. **/
  eeAddress += sizeof(float);                    //Move address to the next byte after float 'f'.
  EEPROM.put(eeAddress, customVar);
  Serial.println("Written custom data type!");

  eeAddress = 100;
  EEPROM.get( eeAddress, f );     // Get the float data from the EEPROM at position 'eeAddress'
  Serial.println( f, 12 );         // This may print 'ovf, nan' if the data inside the EEPROM is not a valid float.

  // get() can be used with custom structures too.
  eeAddress = 100 + sizeof(float); //Move address to the next byte after float 'f'.
  Serial.print("EEPROM address = ");  Serial.println( eeAddress );
  EEPROM.get( eeAddress, customVar );
  Serial.println( "Read custom object from EEPROM: " );
  Serial.println( customVar.field1 );   Serial.println( customVar.field2 );  Serial.println( customVar.name );
}

void loop(){ /* Empty loop */ }

I'm playing with the resolution of the float to see what put and get handle.
 
As a remark I want to point out that get/put only work if your variables can be copied trivially (i.e., bytewise). If you use more complicated objects like e.g. String the get/put code will silently fail.

There is an open pull request https://github.com/PaulStoffregen/EEPROM/pull/5 which is supposed to handle this. I.e. it generates a compile time error if the variable can't be copied to the EEprom. Additionally it makes String variables 'get/put able'.
 
Thanks luni. I'm assuming the Teensy 4.0 represents an internal 32 bit float as IEEE-754 and a bytewise copy will preserve the components of 754. That said..... input strings like $IS,4,0,987654.12345678,23456.65432,3456.6543,4.9876543* do not store and recall at the expected resolution. This may be my misunderstanding of the range of ieee574.
That string is typical of a format I came up with to upload coefficients the Teensy will use in a machine learning application.
I've attached updated code which reads in variable length strings of floats using this
Code:
float freadFromEEPROM(int fp)
{ 
  float floatReturn;
    EEPROM.get(fp, floatReturn);
  return floatReturn;
}


float fwriteToEEPROM(int fp, float floatToWrite)
{
  EEPROM.put(fp, floatToWrite);
  return freadFromEEPROM(fp );  // Return what was written to confirm.
}
the attached code also reads in the float strings from Serial, and fills a float array which is then sent to EEPROM by the code inline above. It's 'working-ish'.
 

Attachments

  • T4_FloatStringsInput.ino
    5.7 KB · Views: 64
All primitive types, i.e. float, double, char int etc. and arrays or structures composed of primitive types and a lot more can be copied bytewise and are suitable for get/put. The code in the linked eeprom.h pull request checks if the type of the passed in variable is bytewise copyable and generates an error if not. If you ever need to check this in your code you can use something like this:

Code:
#include <type_traits>

void setup(){
  while(!Serial);

  String result = "String is";
  if(!std::is_trivially_copyable<String>::value) {
      result += " not ";
  }
  Serial.println( result += "bytewise copyable");

  result = "Float is ";
  if(!std::is_trivially_copyable<float>::value){
      result += "not";
  }
  Serial.println( result += " bytewise copyable");
}

void loop(){
}
Code:
String is not bytewise copyable
Float is  bytewise copyable

I also had a quick look at your code. If I understood correctly, you parse some input string from Serial and transform it to an array of floats. Which, of course can be stored with get/put without issues.

Remark: you might consider strtok() to parse your input string, or even simpler use the arduino String. It doesn't do any harm on a Teensy and makes those things much easier. Here a few examples how to split a string into parts based on separators: https://arduino.stackexchange.com/a/1237/66176
 
Thank You luni,
You are correct about what the code does. It reads a string of comma delimited floats. The parameter after the $IS, is how many floats to expect. The next int (addr) is a sort of array index. The end use is to fill a table with coefficients from a cluster analysis. Thanks for the reminder about strtok(). I'd forgot about it and charged ahead. Will clean things up once 32 floats are going in the out of EEPROM OK.
 
String of floats reader and store to array and EEPROM.

Luni, Pilot, Defragster, et al....

Please let me know if this rabbit-hole adventure is appropriate for the PJRC forum, of if it might be better to move it. Thank You.

The attached .ino is an improved version of the 'float string reader & saver to EERPOM'.

It's "OK-ish" to plop in various sizes of floats to test what the Teensy4.0 stores in the float array and subsequently stores and returns from eeprom. For instance, an input like
Code:
$IS,10,0,1.1,2.2,3.3,4.4,5.5,6.6,7.7,8.8,9.9,987654.456789*
returns the value 987654.43750000 from EEPROM. I think this is an artifact of IEEE754 with float 32's.

But I think I need to wrap a loop around parts of this program and test with a range of very small, and very large positive and negative numbers for general comfort.

Lastly, on the 3.3mS EEPROM write cycle......Do EEPROM.h calls somehow manage this? i.e. should I be inserting delayMicroseconds() after an EERPOM.put & get?

Thank You,
jb
 

Attachments

  • T4_FloatStringsInput.ino
    6.9 KB · Views: 65
This has nothing to do with the eeprom. Float can only store about 6 significant digits. Try double if you really need more
Digits.
 
Well, here is a question. Your application is machine learning. Do you really need that high resolution? A lot of ML takes place on 16- and 32-bit floats.

And another: what is the resolution of whatever this data is coming from? If your input data only goes two digits past the decimal place, but your coefficients (purely due to mathematical artifacts) are much longer, I would consider throwing away the extra. This is exactly what is done in statistical analysis because it avoids implying more precision than is actually there. The standard deviation of a hundred numbers in the format ####.## may look like ##.########, but we know that only the first two digits after the decimal place are actually meaningful, and the rest are nothing more than a side effect of how standard deviation is calculated.
 
Thanks Luni, Right, actually 7.22 decimal places..... https://en.wikipedia.org/wiki/IEEE_754 I'm just looking at the discrepancy cited above. I may have to ignore this small error and move on, which would not cause a significant loss of sleep. ;-)

Thanks Pilot, Oh, I'm sure we have far more resolution than the application requires. I'm just independently exploring the resolution issue for general comfort......there's always the next application. I had the pleasure of working with some fairly fine 24 bit A-D's some years ago and look forward to see what is out there today.
 
Status
Not open for further replies.
Back
Top