512 Channel 16 Bit 1KSPS Data logger Guidance needed!

Status
Not open for further replies.

manasdeore

New member
Hello helpful folks at PJRC Forum!

I am trying to build a 512 Channel 16 Bit Analog Data logger with a sampling speed of 1 Kilo Samples per second per channel.

The analog channels will be read using a parallel bus and I plan to use a Teensy 4.1 for simply reading 16 inputs a total of 512000 times a second and store these values as either a 16bit integer or as hex or as binary in a file in the SD card.

Apart from the ADC values I need to store about 5 analog values and IMU readings for every set of 512 channels of analog inputs.

Here's what I have thought so far - I plan to use 16 numbers of 32 bit analog multiplexers and a 16 channel 16 bit ADC with sampling rate of 1MSPS on parallel bus! I plan to use a standalone Teensy 4.0 just to handle the system status ( Acquiring/ diagnostics etc) and also to control the ADC and Multiplexers. This 4.0 will also give a sync pulse for every sample to the 4.1 to sample the 16 channels and the 4.1 will write a set of samples to the SD card!

Also I am open to recommendations about how I can extract that file from the SD card to lets say a laptop without removing the SD card from the board! I can think of using 2Million baudrate over usb to simply read the files and dump on the serial monitor and then I can have like a program to simple acquire data over serial and post process

These are my questions:-
1) Is the Prop shield pin compatible with teensy4.1? Can I acquire data from the IMU at 1k samples a second? How much time does it take for the current library to acquire one set of IMU data?
2) What is the max SD card size with fastest and reliable rate possible with the Teensy4.1?
3) Am I on the right track with my implementation? Do you think a teensy 4.1 can read 16 inputs 500,000 times a second? Do you think Teensy can also acquire 1K samples of IMU data from the prop shield? Do you think it can also manage to write this data to lets say a 256 GB SD card at such rate ?
4) Any recommended way to extract files from the SD card? Using Ethernet maybe?

What are your thoughts on this?
Any tips/suggestions are welcomed!
 
Hello helpful folks at PJRC Forum!

I am trying to build a 512 Channel 16 Bit Analog Data logger with a sampling speed of 1 Kilo Samples per second per channel.

The analog channels will be read using a parallel bus and I plan to use a Teensy 4.1 for simply reading 16 inputs a total of 512000 times a second and store these values as either a 16bit integer or as hex or as binary in a file in the SD card.

Apart from the ADC values I need to store about 5 analog values and IMU readings for every set of 512 channels of analog inputs.

Here's what I have thought so far - I plan to use 16 numbers of 32 bit analog multiplexers and a 16 channel 16 bit ADC with sampling rate of 1MSPS on parallel bus! I plan to use a standalone Teensy 4.0 just to handle the system status ( Acquiring/ diagnostics etc) and also to control the ADC and Multiplexers. This 4.0 will also give a sync pulse for every sample to the 4.1 to sample the 16 channels and the 4.1 will write a set of samples to the SD card!

Also I am open to recommendations about how I can extract that file from the SD card to lets say a laptop without removing the SD card from the board! I can think of using 2Million baudrate over usb to simply read the files and dump on the serial monitor and then I can have like a program to simple acquire data over serial and post process

These are my questions:-
1) Is the Prop shield pin compatible with teensy4.1? Can I acquire data from the IMU at 1k samples a second? How much time does it take for the current library to acquire one set of IMU data?
I can't answer a lot of things, but I will answer a few that I can.

I'm not sure why you want the prop shield on the Teensy 4.1, unless you want access to the motion sensors or possibly the voltage level shifters for driving WS218B or APA102 lights.

In general, the pins on the prop shield will work with the Teensy 4.1 with the exception that the back 5 pins of the prop shield do not have corresponding holes on the Teensy 4.1. Only one of these pins is used for audio output. The audio output support on the prop shield will NOT work on the Teensy 4.1. The Teensy 4.1 just does not have the digital to analog pin that the Teensy 3.x/LC have to drive a mono sound output. There are other ways to get sound out on the Teensy.

The prop shield has an 8 megabyte flash memory that can be used. But on the Teensy 4.1 you can solder an 8 megabyte ship to the underneath of the Teensy 4.1, and access to this flash memory should be much faster than to the flash memory on the prop shield. You will need the lastest 1.54 beta 9 to properly access this flash memory.

2) What is the max SD card size with fastest and reliable rate possible with the Teensy4.1?

In the released Teensydunio 1.53, the default SD library was limited to 32 gigabytes unless you use the alternate SdFat library. The current betas for 1.54 (currently 1.54 beta 9) have rewritten the SD layer, replacing SD with a thin layer that calls SdFat underneath it.

3) Am I on the right track with my implementation? Do you think a teensy 4.1 can read 16 inputs 500,000 times a second? Do you think Teensy can also acquire 1K samples of IMU data from the prop shield? Do you think it can also manage to write this data to lets say a 256 GB SD card at such rate ?
I don't know on this one.

4) Any recommended way to extract files from the SD card? Using Ethernet maybe?

What are your thoughts on this?
Any tips/suggestions are welcomed!
There are various discussions on exporting SD cards and flash memory file systems to USB support.
 
Hey Michael!

Thank you for your response!
I need the prop shield for the Motion sensors ( The IMU) , accel+gyro to be exact and I thought that since I am going to use the Teensy board anyways why not just use the prop shield on it as it will remove one chunk of hardware from my pcb!

I'll check out the 1.54 Beta 9 for the SD card, Since a 256GB card is quite expensive and would hardly work in any other application, I was hoping to hear a birdie say "IT WORKS!" :p
 
Hello people!

To anyone following this thread, I have successfully tested a 256GB Memory card with Teensy 4.1 using SDFAT library. Hopefully this will help someone in future :)
 
Hello people!

To anyone following this thread, I have successfully tested a 256GB Memory card with Teensy 4.1 using SDFAT library. Hopefully this will help someone in future :)

you should have no problems with 1TB cards (consume more but store more)
 
This is really cool. I wonder if you would be able to do onboard processing of this data, output based off the IMU data processed, and then store it. That's actually very similar to what my plan is.
 
So I am stuck at the logging part of my project! I am currently storing 512 samples of adc in a binary array and I was hoping to figure out a way to simply write them in the sd card!

Here's how I want it to happen, AAAAA<512 samples>YYYYY<MPU Data>ZZZZZ where A, Y and Z are just characters to separate 512 channel data sets, Also I want to limit each file to 500MB and when that file fills upto 500mb, I need the program to create a fresh file and start logging in that.. My problem is that I am not able to figure out how the SDFAT library works, I've tried reading through the unzipped documentation but I am more lost when I go through it!

Can anyone direct me to similar code examples?

Is my approach of writing 512 samples at once a good approach or will I be able to write faster with writing individual data reads to the file? Also blocks are supposed to be uint_8t? My array looks like this - bool ADCBuffer[512][16]

the function that reads the digital pins and stored in the array looks like this - where the readReady variable is set to false by an ISR which is triggered by an external ready pulse
if (!readReady) {
readReady = true;
for (int i = 0; i < 16; i++) {
ADCBuffer[channel] = digitalReadFast(ADC_DB);
}
}

Is there a noobs documentation of the SDFAT library?
 
Suggest to have a look to one of the many data-logger examples discussed on this forum.
you may find something like

Code:
     file.write(diskBuffer,ndat);
ndat is the number of bytes to be written, which for best performance should be a multiple of 512
declaration of diskBuffer does not matter, it will be accessed as "const void *"
 
Status
Not open for further replies.
Back
Top