Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 5 of 5

Thread: Yet another datalogging project

  1. #1
    Junior Member
    Join Date
    Aug 2020
    Posts
    2

    Yet another datalogging project

    Dear community,

    I would like to ask for some advice on a project. I'm trying to log data to a SD card from an ADC and some IMUs running over SPI. I'm logging data for 24 hours. The ADC is sampled at 1 kHz and the IMUs at 100 Hz. I'm using a T4.1 and SdFat-beta. My program is a modification of the ExFatLogger. I have formatted the SD card using the ExFatFormatted before a datalogging session. The ADC spits out 24 bytes of data that needs to read every 1 ms. I'm not to concerned with IMUs as they only need to be read every 10 ms.

    I have managed to "miss" only about 30 seconds of data for the whole 24 hr period by using an old Samsung micro-SD card. So I get 99.96% of the data. I have tried several other micro SD-cards (including SanDisk Industrial and Greenliant using SLC memory) but their performance is worse. It annoys me a bit that I do not get all of the data. I'm going to miss samples when writing to the SD card takes more than 1 ms.

    I'm leaning toward using the DMASPI library for the SPI communication and as long as the DMA buffers are large enough the processor should have enough time to write data to the SD-card. I have not used the DMASPI library or DMA before and the examples for the DMASPI library are a bit difficult to interpret and that is why I'm asking.

    Does anyone have (other) suggestions of how to make this work?

  2. #2
    Senior Member brtaylor's Avatar
    Join Date
    Mar 2016
    Location
    Portland, OR
    Posts
    580
    Can you increase the buffer size?

  3. #3
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    206
    It's a sad fact of life that micro-SD cards often take 100mSec or more to complete a write on the occasions that the internal controller has to erase a block of flash. Since you seem to be collecting about 32KB of data per second, your collection program will occasionally drop 5 to 10 KBytes of data when the SD card is slow to respond. Ideally, you should be collecting data with a timer interrupt routine and putting it into a buffer of at least 16KBytes. The main loop can then read from the buffer and write to the SD card without missing any incoming data.

    This is exactly the kind of problem that the generic data logger object was written to solve. You can find code and discussion in this forum a few months back. Search for "generic datalogger". Your problem is more complex in that you are collecting from two different sensor systems at two different rates. Figuring out how to put them all into one binary file will be a bit of work.

  4. #4
    Junior Member
    Join Date
    Aug 2020
    Posts
    2
    Thanks for the inputs. I have downloaded your Generic Datalogger and tested it a bit. I need to test it a bit more and look more into the code to get a good understanding of how it works.

    Just to make sure I understand. If an interrupt is triggered during the SD-write it will get serviced right away? I was under the impression (most likely incorrectly) that nothing else can happen during the SD write.

  5. #5
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    206
    The SD write routines generally do not mask interrupts for very long (a few microseconds??). Most of the SD write operations are carried out with DMA from file buffers to the SDC hardware. When these DMA operations are complete an interrupt is triggered that sets the completion flags. Depending on the usage, the SD write function may wait for those flags. When I time the SDC writes, most occur in a few tens of microseconds, but there are occasionally much longer times when flash erase is happening. The flash erase is done by the controller in the SD card. While that is happening, the card can buffer a limited number of 512-byte sectors before it has to tell the Teensy to wait.

    For millisecond collection intervals, you can probably assume (but verify) that the interrupt is serviced "right away".

    Take the above statements with a grain of salt. My understanding of the SD write internals may be less than perfect!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •