Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 3 of 3

Thread: Batch Save/Buffering SD Card Help

  1. #1
    Junior Member
    Join Date
    Oct 2019
    Location
    Canada
    Posts
    9

    Batch Save/Buffering SD Card Help

    Hi Guys,
    I need some help understanding data buffering/writing batch data to an SD card. Currently working with the teensy 3.6 and sampling data at 1000 ksps (target). The sd card I bought is Kingston Digital 32GB microSDHC Class 10 UHS-I 100MB/s R 70MB/s W Flash Memory High Speed microSD Card.

    The current code captures data from a fancy DAQ over SPI, and puts it into 27 byte array. This array is converted to hex characters using a an encode function and simply saved in another array called sampleBuffer[55].

    I need to save this sampleBuffer55] successfully without gaps in data, the way i am saving right now is using the SD library and the default read/write example:

    Code:
     String dataString = "";
    
      // Append sampleBuffer to the string, then append millis
    
      dataString += String(sampleBuffer);
      dataString += ",";
      dataString += String(millis());
    
      // open the file. note that only one file can be open at a time,
      // so you have to close this one before opening another.
      wearEMG = SD.open("wearEMG.txt", FILE_WRITE);
    
      // if the file is available, write to it:
      if (wearEMG) {
        wearEMG.println(dataString);
        wearEMG.close();
    This makes a string and appends sampleBuffer as well as millis(), opens a file called wearEMG, writes to the file then closes it. Repeats the process. This obviously is not a good way to save data and is resulting is gaps of about 8ms-12 ms between data, I need to find a clever way to batch samples together(in context of this controller) and batch save them. What would be a good way of doing this? Can I create a second 2d array and save sampleBuffer to it i.e batchSampleBuffer[i][55], write maybe 50-100 samples?...then bulk save those to the sd card?

    Not sure what the best approach would be to something like this. Any advice is appreciate. I have attached a notepad file with the HEX data values.
    Click image for larger version. 

Name:	SD Mode Proof.png 
Views:	3 
Size:	96.3 KB 
ID:	18875

  2. #2
    There are lots of ways to buffer the data. Here is something I did long ago. The data is captured in an interrupt service routine while the foreground task just waits for there to be a full buffer available to write.

    SD card writes are most efficient when done in multiples of 512 bytes. Even so, the SD specification says that a FAT write can take a very long time, up to 700ms, so you need to be able to buffer enough data so that isn't a problem.

  3. #3
    Junior Member
    Join Date
    Oct 2019
    Location
    Canada
    Posts
    9
    Perfect so i should be able to create a buffer with this data, preferably in multiples of 512 bytes, I am assuming this has something to do with sector size? I will try and build a buffer to save a few kilobytes at a time in RAM then write to the SD and see what I have. Your link is very helpful, and the DAQ in interrupt mode while a foreground task to check buffer size and write as necessary sounds like a solid approach. I will give a try. Thanks!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •