Flash - Storing preferences

Status
Not open for further replies.

Frank B

Senior Member
@Paul:

Hello Paul,
on my short trip to the ESP32 world i learned to enjoy the "preferences" library.
It is very handy because it allows to store key/value pairs. This could be a PIN assignment, volume or filter settings, for example. These are retained even when reflashing.
It is similar to a config file, but binary.
So that different programs can use them, there is a parent namspace. For example, the same key can be stored for "Sketch1" and "Sketch2" (and have different values)

"preferences" is the name of the Arduino interface, in the IDF it is called Non-volatile storage library. (link to docu)

My question now: Teensy 4.x has some flash free at the top end.
Can this be used partially? I assume you need less than the 64k.
Is the area partially writable?
Will it be erased when flashing (I hope not) ?

For example, if you could use 48K of it, that would be a practical thing.

If that doesn't work - is there any other way to reserve some memory that is not erased?

Something like EEPROM emulation?
 
LOL, forget some of my questions. It's already used for EEPROM emulation, right?
Ok, then I must be a bit bold and ask if you would spend another 64k for such a feature :)
 
Seems like such a library should be built on top of teensy's EEPROM code. For many uses, far less than 64K is adequate.
 
I could be wrong, but I believe it was in the LittleFS conversation(s), Probably this one: https://forum.pjrc.com/threads/58033-LittleFS-port-to-Teensy-SPIFlash/page12
About the ability to create an FS in Program space:

Code:
#if defined(__IMXRT1062__)
class LittleFS_Program : public LittleFS
{
public:
	LittleFS_Program() { }
In that discussion, if I remember correctly Paul said something along the line that it will take a change to the bootloader chip to preserve the LittleFS FS during reprogramming.

Then again maybe I just dreamed it.... But my guess is that this level of change may be similar.
 
Storing it in the EEPROm makes it useless, as the first write of a program not aware of it (=> all) will kill the data.
But that's the trick: Unlike EEPROM, it can store settings which stay available, even if you run other sketches that save their own settings and are not aware of foreign data.
 
Kurt.. maybe yes.
Years ago, we all said ESP is crap and nothing works. Now, the software and api-side are better than the "old-style" arduino stuff.. so, times are changing...and we should see that.
Yes, that means one should accept that and think about changing some old things.
The questions is, why the bootloader works this way. It would be enough to erase the space needed by the actual program*.

An other nice thing is the esp exception decoder. It is very very helpful when debugging and displays where your program crashed. the source-file, even line-number. incredible useful. it makes debugging fun. I dare not even ask about it..for Teensy.....


*or make that configurable in a way that Arduino does not forget that flag.
 
Last edited:
Ok, then I must be a bit bold and ask if you would spend another 64k for such a feature :)

Probably not another fixed 64K, but....


In that discussion, if I remember correctly Paul said something along the line that it will take a change to the bootloader chip to preserve the LittleFS FS during reprogramming.

Yes, this, more or less.

I'm still not 100% sure of all the details, but my general plan is to make future bootloader versions scan the flash for any LittleFS filesystem and avoid erasing it. The scan will probably start at 512K or maybe 1M and be done at 64K block boundaries. That's why I made a commit to LittleFS_Program to align the filesystem to 64K.

Maybe if we modify LittleFS_Program to also avoid using the last 64K, then that area could become de-facto reserved for a preferences library.

Or maybe the preferences could just be stored as 1 or more files on the LittleFS volume?


Storing it in the EEPROm makes it useless, as the first write of a program not aware of it (=> all) will kill the data.

We could also explore the possibility of some emulated EEPROM addresses being "reserved". The EEPROM emulation would preserve them, and allow them to be read but not written (at least not without a special API used only by the parameter stuff).

But this wouldn't give much storage, maybe a few hundred bytes. This parameter stuff probably wants a model of infrequent writes (low endurance) in exchange for larger storage, right?
 
Oh, I even could imagine a "hidden" file in LittleFS that survives formatting and stores the "preferences".

Depends. It can save "blobs", too. I think the max size of a blob is around 4k (have to read about it..) But yes, it's intended for infrequent writes.
 
Besides the clear need for more structure on top of teensy non-volatile storage, there is the barrier-to-change business issue. It should be easy to port software from ESP32 to teensy. The reverse direction depends on your point of view :).
 
I have thought about it.
I now think it would be far over-engeneered to put it in LittlFS. More disadvantages than advantages.
Placing it in LittleFS would mean that there is an unwanted connection between them. Deleting LittleFS would mean losing all configurations. LittleFS "hacking" to not touch the configuration is more (unnecessary) work and complicates things unnecessarily.
So, please not. Keep both unconnected.

My Question remains. Why gets the flash deleted?
I understand it makes sense on Teensy 3.x. It maybe was a kind of protection against "stealing" code by executing a small stub of code that "prints out" the flash.
For Teensy 4, I have no Idea why it works this way and still deletes all the Flash?. It's dead-easy to just desolder the flash -chip and read it out. Easier than messing with code, jtag etc.
Later, with flash encryption, there are better techniques than deleting everything (I hope ;)



Besides the clear need for more structure on top of teensy non-volatile storage, there is the barrier-to-change business issue. It should be easy to port software from ESP32 to teensy. The reverse direction depends on your point of view :).

Jonr, you're right :)
 
Any easy, standards based way for a PC to access non-volatile data (eeprom or other) in a teensy would be useful. Perhaps virtual files accessed via MTP?

For example, I run audio convolution with large impulse responses. Currently I have to recompile and flash a new program binary just to change the impulse response data. Fine for me, very hard to explain to a non-technical user.

With a backup on PC, I have little concern about unaware programs deleting teensy non-volatile data. Happens on many devices - load incompatible firmware and you lose config data.
 
For ESP, there is a tool that can do this at least for SPIFFS- without reflashing the program.
As it is as python-tool, i think we can use (parts of) it (and I did not look at the lincense, so far)
Search for "ESP Sketch Data Upload".
 
Last edited:
Status
Not open for further replies.
Back
Top