I'm experiencing occasional slow writes to an SD card which are going to interfere with the desired timing in a sensor logging project. It's doing fine on average but some writes are nearly 10X slower, and this seems to be tied to the granularity of the clusters in the Fat 32 formatted card (I used the official SD Formatter 4 from the SD card industry's web page). This shows the timing:
That first really tall peak happened during a line write that took me to 32,488 bytes of my text in the file. There may be some header info that bumps up the total amount of data beyond what I can read in a word processor. That's interestingly close to the cluster size of 32,768. There are smaller events at nearly regular sub-intervals too. When it's been running a while the pattern is strikingly fractal but the upper limit is in the low 200's as seen here. I can imagine that there is some system overhead involved with linking the sectors together in a large file.
There is a practical reason for my concern. My real project involves IMU sensor fusion in addition to the logging and I don't want to compromise either part if I can help it. It would be very cool if I could insure that each cycle would never take more than 50 milliseconds. I intend to trigger the sensor readings, processing, and logging at 50 msec usng an interval timer. There are 11 violations of that timing goal in the interval shown in the chart above. It's not the line of code that does the write that takes the time, it's the file closing. I tried another approach and got the same results: only open the file once, never close it, and use the flush() function to make the writes happen. Without either the close() or the flush(), the data never gets into the card. I'm using the standard SD library, and the Adafruit SD card breakout. Processor is the Teensy 3.1.
Here are some options I have:
1) Speed things up in some way that I don't yet know
2) Make the writes more consistent, in some way I don't know
3) Multitask: Delegate the SD logging to an additional Teensy 3.1 (the average time is around 30 msec) but it would need an input buffer of nearly 1K
4) Just accept it
I know how to do options 3 and 4, but options 1 or 2 would preferable. Thanks in advance for any suggestions that may be contributed.
Here is the representative sample code that demonstrates the issue:
That first really tall peak happened during a line write that took me to 32,488 bytes of my text in the file. There may be some header info that bumps up the total amount of data beyond what I can read in a word processor. That's interestingly close to the cluster size of 32,768. There are smaller events at nearly regular sub-intervals too. When it's been running a while the pattern is strikingly fractal but the upper limit is in the low 200's as seen here. I can imagine that there is some system overhead involved with linking the sectors together in a large file.
There is a practical reason for my concern. My real project involves IMU sensor fusion in addition to the logging and I don't want to compromise either part if I can help it. It would be very cool if I could insure that each cycle would never take more than 50 milliseconds. I intend to trigger the sensor readings, processing, and logging at 50 msec usng an interval timer. There are 11 violations of that timing goal in the interval shown in the chart above. It's not the line of code that does the write that takes the time, it's the file closing. I tried another approach and got the same results: only open the file once, never close it, and use the flush() function to make the writes happen. Without either the close() or the flush(), the data never gets into the card. I'm using the standard SD library, and the Adafruit SD card breakout. Processor is the Teensy 3.1.
Here are some options I have:
1) Speed things up in some way that I don't yet know
2) Make the writes more consistent, in some way I don't know
3) Multitask: Delegate the SD logging to an additional Teensy 3.1 (the average time is around 30 msec) but it would need an input buffer of nearly 1K
4) Just accept it
I know how to do options 3 and 4, but options 1 or 2 would preferable. Thanks in advance for any suggestions that may be contributed.
Here is the representative sample code that demonstrates the issue:
Code:
// SD Card Stuff
#include <SPI.h>
#include <SD.h>
File logfile;
char filename[13];
const int slaveSelectPin = 10;
char SDline[300];
// Globals
long afterclose = 0;
long tstart;
/*******************************************
* setup()
*******************************************/
void setup()
{
pinMode(3, INPUT_PULLUP);
Serial.begin(115200);
// SD Card File Naming and Creation
pinMode (slaveSelectPin, OUTPUT);
if (!SD.begin(slaveSelectPin)) {
Serial.println("Is an SD card in place?");
}
// Construct a unique file name
strcpy(filename, "SDCARD00.csv"); // So we know what's in the file when looking at the file list
for (uint8_t i = 0; i < 100; i++) {
filename[6] = '0' + i/10;
filename[7] = '0' + i%10;
if (! SD.exists(filename)) { // create if does not exist, do not open existing, write, sync after write
break;
}
}
// Get ready to log
logfile = SD.open(filename, FILE_WRITE);
if(! logfile ) {
Serial.println("Couldn't open the file on the SD card; stopping now.");
while(1); // might as well stop if we can't open the file for logging
}
Serial.println(filename); // If we got this far we were successful with the SD file.
logfile.println("msec C, text1, text2, msec A, msec B");
logfile.close();
}
/*******************************************
* loop()
*******************************************/
void loop()
{
tstart = millis();
char ctmp[12];
strcpy(SDline, ""); // start fresh each time
logfile = SD.open(filename, FILE_WRITE);
itoa(int(afterclose), SDline, 10); // Initiate the line with the "msec C" data obtained from the previous iteration of loop()
strcat(SDline, ",");
strcat(SDline, "a012345678901234567890123456789012345678901234567890123456789,"); // just some generic text to bring up the character count
strcat(SDline, "b012345678901234567890123456789012345678901234567890123456789,"); // to simulate the sensor data and calculation results
// msec A
strcat(SDline, itoa(int(millis()-tstart), ctmp, 10));
strcat(SDline, ",");
// msec B
strcat(SDline, itoa(int(millis()-tstart), ctmp, 10));
strcat(SDline, ","); // Last item!
logfile.println(SDline); // Log it
logfile.close();
// msec C; this is where the timing variability occurs
// THE FILE IS ALREADY CLOSED; we'll have to log it at the start of the next line
afterclose = millis()-tstart; // we'll print it at the head of the next line, once we get into the next cycle of loop()
}
Last edited:
