Batch Save/Buffering SD Card Help

Status
Not open for further replies.

bk2001050

Member
Hi Guys,
I need some help understanding data buffering/writing batch data to an SD card. Currently working with the teensy 3.6 and sampling data at 1000 ksps (target). The sd card I bought is Kingston Digital 32GB microSDHC Class 10 UHS-I 100MB/s R 70MB/s W Flash Memory High Speed microSD Card.

The current code captures data from a fancy DAQ over SPI, and puts it into 27 byte array. This array is converted to hex characters using a an encode function and simply saved in another array called sampleBuffer[55].

I need to save this sampleBuffer55] successfully without gaps in data, the way i am saving right now is using the SD library and the default read/write example:

Code:
 String dataString = "";

  // Append sampleBuffer to the string, then append millis

  dataString += String(sampleBuffer);
  dataString += ",";
  dataString += String(millis());

  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  wearEMG = SD.open("wearEMG.txt", FILE_WRITE);

  // if the file is available, write to it:
  if (wearEMG) {
    wearEMG.println(dataString);
    wearEMG.close();

This makes a string and appends sampleBuffer as well as millis(), opens a file called wearEMG, writes to the file then closes it. Repeats the process. This obviously is not a good way to save data and is resulting is gaps of about 8ms-12 ms between data, I need to find a clever way to batch samples together(in context of this controller) and batch save them. What would be a good way of doing this? Can I create a second 2d array and save sampleBuffer to it i.e batchSampleBuffer[55], write maybe 50-100 samples?...then bulk save those to the sd card?

Not sure what the best approach would be to something like this. Any advice is appreciate. I have attached a notepad file with the HEX data values.
SD Mode Proof.png
 
There are lots of ways to buffer the data. Here is something I did long ago. The data is captured in an interrupt service routine while the foreground task just waits for there to be a full buffer available to write.

SD card writes are most efficient when done in multiples of 512 bytes. Even so, the SD specification says that a FAT write can take a very long time, up to 700ms, so you need to be able to buffer enough data so that isn't a problem.
 
Perfect so i should be able to create a buffer with this data, preferably in multiples of 512 bytes, I am assuming this has something to do with sector size? I will try and build a buffer to save a few kilobytes at a time in RAM then write to the SD and see what I have. Your link is very helpful, and the DAQ in interrupt mode while a foreground task to check buffer size and write as necessary sounds like a solid approach. I will give a try. Thanks!
 
Status
Not open for further replies.
Back
Top