Logging Storage Options

Status
Not open for further replies.

DemolishunWork

Active member
I am working on a project that requires some data to be logged on a Teensy 3.1. What are some simple options for storing data?

One of my constraints is I cannot spend very much time writing data to a device. So whatever I write to cannot block my application for very long. I read that SD cards can cause the app to block for long periods of time. So I am wary of that option. I read something about "SPI mode" or SDIO mode. Not sure which is which, but people seamed to indicate that SDIO mode is faster. I also thought about using the internal flash on the device, but that would limit me to 256K - minus my apps storage space. Not sure that will be enough.

I am thinking maybe high capacity EEPROM or some other technology. Also, whatever device I use may end up being exposed to 300F for long periods of time (not even sure I can use a Teensy 3.1 at that point). I also realize because of that fact I may end up redesigning multiple times. I know other people will test components actively at various high temps to determine what devices pass and fail under those conditions.

So any thoughts on fast storage methods would be awesome, thanks.
 
I recently did some testing with the audio library Recorder example. As you can see on line 151, the actual time taken to write a 512 byte sector is printed by those 2 lines.

You can think of this code as logging 44100 16 bit values per second. The audio library implements a queue, so new data isn't lost if the SD card latency is less than the queue length (approx 170 ms in that example) and if the average latency is low enough after occasional higher latency, so the queue drains before another high-latency write occurs.

Here's the write times for 3 different SD cards I tested:

SanDisk Ultra 16 GB:

Code:
SD write, us=1476
SD write, us=1475
SD write, us=10692
SD write, us=7370
SD write, us=6812
SD write, us=1437
SD write, us=1427
SD write, us=1430
SD write, us=1447
SD write, us=1435
SD write, us=1457

SanDisk (generic) 8GB:

Code:
SD write, us=1435
SD write, us=4199
SD write, us=1516
SD write, us=1423
SD write, us=1451
SD write, us=1427
SD write, us=1461
SD write, us=1433

(very old) Transcend 128 MB:

Code:
SD write, us=5284
SD write, us=7102
SD write, us=10068
SD write, us=4811
SD write, us=5911
SD write, us=8421
SD write, us=8177
SD write, us=6035
 
While some SD cards might be worse, you can probably plan on needing to buffer about 25 ms of data while the SD card writes, if you're sampling faster than 40 Hz.

Teensy 3.1 has plenty of memory for buffering, but you do need to use interrupts or some other way to collect data while the SD card is busy.

You didn't say how fast your data actually is, but hopefully that audio recording example can serve as a benchmark showing 44100 samples per second at 16 bits each is pretty achievable, with good software design.
 
Wow! Some really great options here.

@onehorse,
Those EEPROMs are very nice!

@Paul,
I also like the idea of being able to use an SD card so that is a great surprise. My data will probably be 5 to 10 Hz, perhaps less.

I still need to calculate the storage capacity I will need. Right now the sensors I will be using and the rate at which I need to take the data is not defined. Once I get that defined I can work the numbers to determine total storage. The nice thing is the data is elevation related so I don't store data unless the device moves. We will be logging some events as well, but that is generally when it is "active" or moving.

Is there an add on board for Teensy 3.1 to add an SD card? I searched around, but was finding one for the older Teensy boards.
 
At only 5 to 10 Hz, unless you're logging more than 512 bytes per event, your data rate will be so slow that SD card latency shouldn't be an issue.

But if you did need to go faster, with some buffering and good software design, much faster speeds are achievable!
 
Status
Not open for further replies.
Back
Top