Forum Rule: Always post complete source code & details to reproduce any issue!
Page 2 of 2 FirstFirst 1 2
Results 26 to 31 of 31

Thread: LittleFS performance issue for QSPI flash from 1.54 beta10 to 1.56?

  1. #26
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,196
    As far as p#23 - doing the formatUnused process 'at some non time critical point' will prevent the format on use slowdown as the dirty blocks have to be used again.

    Also if files are migrated off and deleted, that would be a good time to format dirty blocks.

  2. #27
    I do not know what LFS is doing under the covers with the directory structure when files are opened and appended, but my use case is pretty simple. I am opening, writing (appending) 150 bytes, and then closing a file over and over. Preferably 50-100 times a second. This is to log telemetry data on an amateur rocket. So, I am not creating a lot of files, but I am updating the same file over and over. Little FS is extremely fast at writing if I do not close the file or flush after each write, but it is critical that I don't lose milliseconds of data, because when something doesn't go right (boom!) the most important data is usually in the last second -- like a black box on an airplane. With 1.55 when I low level format the open-write(append)-close rate is about 98 per/sec then it slows after about 1Mb of writing and drops down to about 10 per/sec. in TD1.56 it is even slower. It slows rapidly to only about 2-3 per/second. My work-around is to not use open-close and instead just open at the start and set a timer to flush() every 1000ms. That way it is very fast (500+ per/sec) with a small pause to flush. Only one second of data is "at risk" of not getting physically written. This will work for me for now. If there is a way to improve the open-close speed of LFS in the future that would be great. Thanks for everything you are putting into this platform. I love it!

  3. #28
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,196
    In order to commit data on LFS it must finalize a write once block - then that block has to be reformatted. Any updates are done to a freshly formatted block.

    Doing that 50 to 100 times per second the update to confirm 150 bytes written will take a new data block for the end of data of the file and an update to the directory block, forcing it to use a fresh block. So that causes at least two blocks to be 'trashed' { not sure where the link to that block is stored - that may require update to a third block }

    This will quickly trash hundreds of blocks per seconds with the updates.

    Not sure of actual total capacity needed?: "a few minutes when the chip has about 2Mb of data written"

    Actual data is only 11250 bytes per second at ~75 avg updates per second. Not sure how long a flight lasts in total? But about 88 seconds would fit into one Megabyte - so perhaps better to use FRAM/Magnetic Ram that isn't very large - costs more - but maintains data and doesn't have such reformat overhead AFAIK.

  4. #29
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    8,273
    Quote Originally Posted by TeensyDigital View Post
    ....... I am opening, writing (appending) 150 bytes, and then closing a file over and over. Preferably 50-100 times a second. This is to log telemetry data on an amateur rocket. So, I am not creating a lot of files, but I am updating the same file over and over. Little FS is extremely fast at writing if I do not close the file or flush after each write, but it is critical that I don't lose milliseconds of data, because when something doesn't go right (boom!) the most important data is usually in the last second -- like a black box on an airplane. ....
    It is interesting that @defragster brought up using FRAM instead of FLASH. Right now you can't use a FRAM chip with QSPI as far as I remember only SPI. It is probably a bit slower but you can still use LFS with it.

    You might want to check this article out on Model Rocket flight recorder:
    https://nutsvolts.texterity.com/nuts...=38&pg=38#pg38
    and
    https://nutsvolts.texterity.com/nuts...=41&pg=41#pg41

  5. #30
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,196
    As @mjs513 notes the FRAM interface isn't as best case fast using standard SPI, though net throughput numbers not at hand just now.

    And looking at the supported device table seems 1MB is the largest listed?: //Cypress 8Mb FRAM, CY15B108QN

  6. #31
    yep. I looked at the FRAM earlier, but it is very small. I would have to do a lot of refactoring and slow my data rates down. I am spoiled with 64Mb of logging now. I used to used actual SD cards, but those are only about 90% reliable in a rocket (vibration, g forces, impacts, etc.). I collect data at a slower rate when it is on the pad and after it has landed, but sometimes it can sit on the pad for an hour. The high resolution stuff is during flight and descent. I have been using the QSPI flash for over a year a dozens of launches and it has been very reliable, until the recent performance issues forced me to look closer. There is a lot of overhead, but so far I think I have it working. I am now doing long-term tests at 396 Mhz to see if I can slow my board down to keep it slightly cooler. Launching in the Mojave desert in July is like putting it in an oven with ambient temps of 65C+.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •