LittleFS performance issue for QSPI flash from 1.54 beta10 to 1.56?

As far as p#23 - doing the formatUnused process 'at some non time critical point' will prevent the format on use slowdown as the dirty blocks have to be used again.

Also if files are migrated off and deleted, that would be a good time to format dirty blocks.
 
I do not know what LFS is doing under the covers with the directory structure when files are opened and appended, but my use case is pretty simple. I am opening, writing (appending) 150 bytes, and then closing a file over and over. Preferably 50-100 times a second. This is to log telemetry data on an amateur rocket. So, I am not creating a lot of files, but I am updating the same file over and over. Little FS is extremely fast at writing if I do not close the file or flush after each write, but it is critical that I don't lose milliseconds of data, because when something doesn't go right (boom!) the most important data is usually in the last second -- like a black box on an airplane. With 1.55 when I low level format the open-write(append)-close rate is about 98 per/sec then it slows after about 1Mb of writing and drops down to about 10 per/sec. in TD1.56 it is even slower. It slows rapidly to only about 2-3 per/second. My work-around is to not use open-close and instead just open at the start and set a timer to flush() every 1000ms. That way it is very fast (500+ per/sec) with a small pause to flush. Only one second of data is "at risk" of not getting physically written. This will work for me for now. If there is a way to improve the open-close speed of LFS in the future that would be great. Thanks for everything you are putting into this platform. I love it!
 
In order to commit data on LFS it must finalize a write once block - then that block has to be reformatted. Any updates are done to a freshly formatted block.

Doing that 50 to 100 times per second the update to confirm 150 bytes written will take a new data block for the end of data of the file and an update to the directory block, forcing it to use a fresh block. So that causes at least two blocks to be 'trashed' { not sure where the link to that block is stored - that may require update to a third block }

This will quickly trash hundreds of blocks per seconds with the updates.

Not sure of actual total capacity needed?: "a few minutes when the chip has about 2Mb of data written"

Actual data is only 11250 bytes per second at ~75 avg updates per second. Not sure how long a flight lasts in total? But about 88 seconds would fit into one Megabyte - so perhaps better to use FRAM/Magnetic Ram that isn't very large - costs more - but maintains data and doesn't have such reformat overhead AFAIK.
 
....... I am opening, writing (appending) 150 bytes, and then closing a file over and over. Preferably 50-100 times a second. This is to log telemetry data on an amateur rocket. So, I am not creating a lot of files, but I am updating the same file over and over. Little FS is extremely fast at writing if I do not close the file or flush after each write, but it is critical that I don't lose milliseconds of data, because when something doesn't go right (boom!) the most important data is usually in the last second -- like a black box on an airplane. ....

It is interesting that @defragster brought up using FRAM instead of FLASH. Right now you can't use a FRAM chip with QSPI as far as I remember only SPI. It is probably a bit slower but you can still use LFS with it.

You might want to check this article out on Model Rocket flight recorder:
https://nutsvolts.texterity.com/nutsvolts/200708/?folio=38&pg=38#pg38
and
https://nutsvolts.texterity.com/nutsvolts/200709/?folio=41&pg=41#pg41
 
As @mjs513 notes the FRAM interface isn't as best case fast using standard SPI, though net throughput numbers not at hand just now.

And looking at the supported device table seems 1MB is the largest listed?: //Cypress 8Mb FRAM, CY15B108QN
 
yep. I looked at the FRAM earlier, but it is very small. I would have to do a lot of refactoring and slow my data rates down. I am spoiled with 64Mb of logging now. I used to used actual SD cards, but those are only about 90% reliable in a rocket (vibration, g forces, impacts, etc.). I collect data at a slower rate when it is on the pad and after it has landed, but sometimes it can sit on the pad for an hour. The high resolution stuff is during flight and descent. I have been using the QSPI flash for over a year a dozens of launches and it has been very reliable, until the recent performance issues forced me to look closer. There is a lot of overhead, but so far I think I have it working. I am now doing long-term tests at 396 Mhz to see if I can slow my board down to keep it slightly cooler. Launching in the Mojave desert in July is like putting it in an oven with ambient temps of 65C+.
 
@OP did you ever get this sorted?

I have recently moved a datalogging system from SD cards to an SPI flash chip, and my write times for the chip using LittleFS for Open/save/close for around 70 bytes is anywhere from 300ms to 2000 ms (yes 2 seconds). I too have a 64 mbit chip.

I fixed my issue by writing my own database driver that saves data to the chip using classic fields and records. I save data every second for some 2 hours and my save times per record (which are about 57 bytes) are 1.6 milliseconds--yea--this lib is fast. There are no open/save/close hit times and I kinda have provisions for writing to recordset A, then recordset B, then more to recordset A. It does not have a fat table or overhead to slow things down. Also records are a fixed number of fields with fixed data types. You can specify both during setup, but you can't change during loop. Remember DBase III from the 80's? I used to program in that language and leveraged much of those concepts.

I have also added things like gotoRecord() and getField() functions so I can suck the data off the chip and write to and SD card (post race processing, where I can open/write, write, write.../close.

So far it's very reliable and I'm happy to share. I do have some more chores to with it....
 
My issue was very specific to the time/overhead it takes to close or flush (commit) after writing. My design point is to log a line 100 times a second or faster with a 100% commit rate to get high resolution data on a rocket launch. The 100% commit rate is important, because vehicles often cato (go catastrophic) in a matter of milliseconds. The logged data provides a type of "black box" for post mortem analysis. Unfortunately, the best I can do with LittleFS is keep the file open, log fast, and flush (commit) once a second. This gives me a high rate of logging, but also gives me big gaps once a second in my data (can get as high as 200ms) and I always risk losing the final second of data in a launch that ends poorly. I have not looked at this, since last summer, so it is possible new versions are improved.
 
@OP did you ever get this sorted?

I have recently moved a datalogging system from SD cards to an SPI flash chip, and my write times for the chip using LittleFS for Open/save/close for around 70 bytes is anywhere from 300ms to 2000 ms (yes 2 seconds). I too have a 64 mbit chip.

I fixed my issue by writing my own database driver that saves data to the chip using classic fields and records. I save data every second for some 2 hours and my save times per record (which are about 57 bytes) are 1.6 milliseconds--yea--this lib is fast. There are no open/save/close hit times and I kinda have provisions for writing to recordset A, then recordset B, then more to recordset A. It does not have a fat table or overhead to slow things down. Also records are a fixed number of fields with fixed data types. You can specify both during setup, but you can't change during loop. Remember DBase III from the 80's? I used to program in that language and leveraged much of those concepts.

I have also added things like gotoRecord() and getField() functions so I can suck the data off the chip and write to and SD card (post race processing, where I can open/write, write, write.../close.

So far it's very reliable and I'm happy to share. I do have some more chores to with it....

Hi Kris, that sounds like an interesting setup for the datalogger. And yes think folks will be interested.
 
Thought I'd revive this old thread.
I just tested the benchmark sketch in the first post. This is my result. Is it good or bad?
Code:
Completed 100 open/write/close in a total of  26845 ms - avg tx/ms = 268
Per Tx in Micros:  Avg open: 6950    Avg print: 258151    Avg close: 3345
Completed 100 open/write/close in a total of  28894 ms - avg tx/ms = 288
Per Tx in Micros:  Avg open: 7675    Avg print: 277874    Avg close: 3387
Completed 100 open/write/close in a total of  32417 ms - avg tx/ms = 324
Per Tx in Micros:  Avg open: 5308    Avg print: 318475    Avg close: 392
Completed 100 open/write/close in a total of  34233 ms - avg tx/ms = 342
Per Tx in Micros:  Avg open: 5792    Avg print: 333106    Avg close: 3424
Completed 100 open/write/close in a total of  35521 ms - avg tx/ms = 355
Per Tx in Micros:  Avg open: 6528    Avg print: 345205    Avg close: 3476
Completed 100 open/write/close in a total of  24783 ms - avg tx/ms = 247
Per Tx in Micros:  Avg open: 7267    Avg print: 237135    Avg close: 3426
Completed 100 open/write/close in a total of  26428 ms - avg tx/ms = 264
Per Tx in Micros:  Avg open: 6792    Avg print: 257087    Avg close: 396
Completed 100 open/write/close in a total of  30422 ms - avg tx/ms = 304
Per Tx in Micros:  Avg open: 5381    Avg print: 295384    Avg close: 3457

Reason I ask is because I have a 16mb external flash, I then use the wonderful FlasherX to recieve a .hex file over Serial from a Bluetooth UART interface.
The Bluetooth is actually quite slow at this point, takes about 10 seconds to transfer 1222 lines (blink sketch). Even this is way to fast for the LittleFS.,
What I do is open a .txt file in LittleFS and the incoming data from Serial is saved in that text file.
I see the output from Serial slow down and when I open the file, data is missing.

To test the "max speed" I put a blink sketch as an array, I then take that data and write into a .txt file in LittleFS.
See the attached video of the insanely slow speed. Note that there are no delays. Using LittleFS and MTP (So that I later on can access the file on the PC).

Video: https://streamable.com/e/4ypwuh

Here is the main parts of the code if anyone is interested.
This slow speed can't surely be the real limit? :eek:

Code:
size_t countHexLines(const char* progmemString) {
  size_t count = 0;

  for (size_t i = 0; pgm_read_byte(&progmemString[i]) != '\0'; i++) {
    if (pgm_read_byte(&progmemString[i]) == ':') {
      count++;
    }
  }

  return count;
}

const char* getNextLine(const char* largeArray, size_t OTA_arraySize) {
  static const char* currentPosition = nullptr; // Static variable to keep track of the current position
  
  // Initialize currentPosition on the first call
  if (currentPosition == nullptr) {
    currentPosition = largeArray;
  }

  // Check if currentPosition is still within the array bounds
  if (currentPosition >= largeArray + OTA_arraySize) {
    // End of array reached
    currentPosition = nullptr; // Reset for future calls
    return nullptr;
  }

  // Find the end of the current line
  const char* endOfLine = currentPosition;
  while (endOfLine < largeArray + OTA_arraySize && *endOfLine != '\n') {
    endOfLine++;
  }

  // Calculate the length of the line
  size_t lineLength = endOfLine - currentPosition;

  // Create a buffer for the line (including null terminator)
  static char lineBuffer[256]; // Adjust buffer size as needed
  memset(lineBuffer, 0, sizeof(lineBuffer));

  // Copy the line into the buffer
  strncpy(lineBuffer, currentPosition, lineLength);

  // Move currentPosition to the beginning of the next line
  currentPosition = endOfLine + 1;

  // Return a pointer to the line buffer
  return lineBuffer;
}

bool stop = false;
void test_performUpdate() {
  MTP.loop();
  if (!stop) {
    const char* currentLine;

    if (OTA_TotalLines == 0) {
      if (firmware_buffer_init(&OTA_buffer_addr, &OTA_buffer_size) == 0) {
        Serial.printf("unable to create buffer\n");
        for (;;) {}
      }
      
      Serial.printf("created buffer = %1luK %s (%08lX - %08lX)\n", OTA_buffer_size/1024, IN_FLASH(OTA_buffer_addr) ? "FLASH" : "RAM", OTA_buffer_addr, OTA_buffer_addr + OTA_buffer_size);

      OTA_arraySize = sizeof(hexString) - 1;
      OTA_TotalLines = countHexLines(hexString);
    }

    if (OTA_Counter <= OTA_TotalLines) {
      currentLine = getNextLine(hexString, OTA_arraySize);
      
      File dataFile = myfs.open("test.txt", FILE_WRITE);
      if (dataFile) {
        dataFile.println(currentLine);
        Serial.println(currentLine);
        dataFile.close();
      }

      //custom_update_firmware(currentLine, serial, OTA_buffer_addr, OTA_buffer_size);
      OTA_Counter++;
    } else {
      stop = true;
  }
}
 
Last edited:
Back
Top