The strange SD card behavior

[Yannick]

Hello all,

Trying to do a data logger, I used Greiman's "TeensySdioDemo.ino" to test File writing with a teensy 3.6 on a 8GB SanDisk Ultra and a 8GB low cost one (I do not remember where it comes from).
I dont need high average speed but continuous writing speed, and therefor I whished to estimate the duration of the busy wait state / write time of the SD, which can induce data lost...
So I modified a litle bit "TeensySdioDemo.ino" to plot the write time for each 32kB block along the file write.
"run test" writes N files of N blocks of N Bytes , each "N" being choosen by the user.


In summary :

- the type of SD card is very very important... (not new)
On the SanDisk busy periods where more frequent but shorter than on the "low cost" card.

- A 10 milli-second pause between block write, improves the write speed of the blocks (not of the file) or decrease busy states occurence/duration.

- I do not find any improvement using pre-allocating/erasing. This said I am not sure to have correctly used the code of "tni" to pre-allocate, pre-erase the file space. "my" durty code is there under attached. If lines 62..78 are wrong, please help me...
View attachment K66-TeensySdioDemo.ino


A few fig of my trials : ( Milli-Seconds in ordinate / 32kBytes Block Number in abscissa. )
I am interested in any Ref of good SD cards...
Thank's in advance. y.


SanDisk not preallocated


SanDisk preallocated : a little bit cleaner at the beginning?


SanDisk not preallocated but a 10mS pause between 32kB block write : Small busy states disappeared...



LowCost not preallocated


LowCost preallocated : I could not allocate more than 4096 blocks on this card, anyway no clear improvment... ( X scale changed )


LowCost not preallocated but a 10mS pause between 32kB block write : the big spike disappeared ( Y scale changed )

 

[UhClem]

You never specify what data rate you require.

As a data point, I record at 256KB/sec with no trouble. But I skipped a file system and libraries.

 

[Yannick]

Thank's UhClem,

32kHz * 8 channels * 2Bytes -> 512kB/S data rate would be perfect...
A 192kB circular buffer would be full after 192/512= 0.375 Secondes acquisition time, or less if I add a time stamp...
Some cards freezes more, so I look for the best way to use these SD cards...
Else I will resign to lost some data...

What do you mean by : " But I skipped a file system and libraries. " ?

Y.

 

[UhClem]

Thank's UhClem,

32kHz * 8 channels * 2Bytes -> 512kB/S data rate would be perfect...

Shouldn't be a problem unless the file system causes trouble. Imagine what would happen if when allocating a new cluster it had to skip over a large group of clusters already in use.

A 192kB circular buffer would be full after 192/512= 0.375 Secondes acquisition time, or less if I add a time stamp...

Write performance improves with larger buffers. If you used six 32KB buffers that would mean while writing one you had five more for data. So there is a tradeoff here. Smaller buffers means you have more usable buffer space in the worst case but lower write speeds. I can think of two ways to work around this but the marginal gain probably isn't worth the effort.

What do you mean by : " But I skipped a file system and libraries. " ?

Forth

 

[Yannick]

Thank's again UhClem,

Forth is another story...

In fact I wanted to have a clear idea on the characteristics of the SD cards, best refs..., and of the efficiency of the library.

The best test to answer my concerns is the code of TNI : https://github.com/tni/teensy-samples/blob/master/SdFatSDIO_low_latency_logger.ino
I played with it and when I run it with a total of 192kB buffers and capture data = 16 Bytes at 100kHz, that is 3 times faster than what I really need, everything is OK, no overflow... (Only with my SanDisk card, the other one refuses to be erased!!!)
Thus I do not worry too much.
Y.