Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 3 of 3

Thread: Should I split Arduino/Teensy SD log files to prevent a total file corruption?

  1. #1
    Junior Member
    Join Date
    Jan 2019
    Posts
    1

    Should I split Arduino/Teensy SD log files to prevent a total file corruption?

    Currently I am developing a system which will work under extreme situations (amateur rocketry!), and an old question has risen again:

    Should I split my data log files in parts, to prevent a total corruption of a single file?

    Ex:

    1) Instead of just having

    "log.bin", of 1024KiB;

    2) Split the file as

    "log00.bin", of 128KiB,

    "log01.bin", of 128KiB,

    ...

    "log07.bin", of 128KiB.

    Using a Teensy 3.6, SdFatSdioEx object from SdFat library.

    Thanks!

  2. #2
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    5,020
    I'd use a flash chip. No error-prone connectors.

  3. #3
    Senior Member xxxajk's Avatar
    Join Date
    Nov 2013
    Location
    Buffalo, NY USA
    Posts
    490
    Most of the FAT implementations use Chan's rather broken library.
    I too use it, but have actually fixed it to work properly.
    If you were to use the UHS3 implementation, which now supports SDcard on SPI, you might not have the corruption issues.
    Yes, this won't help t3's built-in sdcard, but, you can always use an SPI card reader.

    I might port the t3's sdcard stuff, but it won't happen any time soon.
    If you want to contribute to support it, all that is required is to read and write blocks and detect when the sdcard is inserted. UHS_FS does all of the remaining tasks that would be involved.

    UHS3 has many abilities to help avoid file corruption easily, so, basically you could do one of two things:
    1. After each write close and then re-open the file and append to it (can other libraries do this??), or open another file. This has a disadvantage where it has to do a lot of work, and is slow.
    2. Using UHS3, have UHS_FS to sync everything that is out of date after a write. fs_sync(); does this, and it is pretty fast, since only dirty buffers are written and the FAT tables are synchronized.

    Nothing is going to be able to totally prevent corrupted data if you are mid-way in a write operation to the media, no matter what lib you use..
    You should probably have a button to press that can sync the filesystem and unmount the sd media properly. Not sure how or even if the other libs can actually do that.
    UHS_FS during an unmount closes all open files, and synchronizes the media for you.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •