Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 12 of 12

Thread: Maximum average throughput to SD card on T4.1

  1. #1

    Maximum average throughput to SD card on T4.1

    Hi All

    I have an application where I am capturing 4 channels of 24bit Analog at 128ksps.

    The T4.1 I am using has 16MBytes of QSPI ram installed and I use this as a buffer for batch mode but intend to use some of it as a circular buffer for a continuous write to SD card if it can be made to write fast enough on average.

    Simple testing so far shows that I can get an average write speed (when just writing a batched file from RAM to card) of about 2Mbytes per second.

    With the above scheme operating, it looks like I will need 128000 * (4 x 3 (ADC)+ 8 (digital inputs) ) = 2560000 bytes per second so faster than I have observed the SD writes on average to date.

    If I trim the ADC results to 16 bit (realistically that's all the resolution this ADC can achieve at that sample rate) then this becomes 128000x (4x2 + 8) = 2048000 bytes per second so might be doable.

    Does anyone know if an SD card write on the T4.1 with plain SPI can write this fast on average?

    Has anyone tried using one of the QSPI ports to support an SD card interface?


    Craig

  2. #2
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,708
    The T_4.1 on board SD socket is QSPI interface - actually SDIO 4 bit.
    Using latest TeensyDuino 1.57 the SD code includes an included version of SdFat that gives much better I/O speed for reads and even writes.
    There are examples included that will give sample transfer speed notes IIRC.
    The speed can be influenced by the SD Card in use - there are notes on the SD Card type generally seen as best ... and A1 or A2 support is better as noted by Paul.

  3. #3
    Hi Defragster, that is awesome news!

    I'll upgrade my TeensyDuino to latest today and revise what I am getting.

    Many thanks for your feedback.

  4. #4
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    27,395
    To achieve fast write speeds, you will need to collect data into buffers in memory and write in fairly large chunks. 4096 bytes per write should work well with A1 & A2 cards. Larger sizes are needed for normal cards.

  5. #5
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,473
    The T4.1 build-in SD card can easily handle 4 channels at 32 bit at 192 kHz sampling (more than you asked). External SPI SD cards are much more limited.
    Issue remains, that you need to prepare a huge buffer that can hold at least 150 ms of data to bridge the infrequent but real SD card internal business.

  6. #6
    Maximum write speed requires writing data in RU (Recording Unit) sized blocks which are over 128KB for big SD cards.

    But not to worry, it has been a while but I recall looking at the code for one library (I forget which one) and rather than complete a small write would leave it open. Continuing with the next write call if it was contiguous. Achieving very large write sizes even the if caller didn't do it explicitly.

    The real hassle is the delays imposed by the FAT file system. (If used) The SD specification includes a 750ms max FAT write time which includes file system overhead. If you run into that you must have sufficient FIFO buffer space or you will lose data.

    With the Teensy you will want to check the data sheets for the SD port and make sure that you choose a clock configuration that allows for the best SD clock speed. Integer dividers are the limit here.

  7. #7
    Thanks all,

    Very useful information.

    I have a 16MByte ram buffer so should be covered for SD write delays..

    Will have a play with this using various sized 'chunks' to see what effect that has on performance.

  8. #8
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,708
    Using this included example ref'd in p#2: {?}\hardware\teensy\avr\libraries\SdFat\examples\b ench\bench.ino

    This card not just formatted - no idea when that was ...
    Going from 512B to 4096B buffer pushes write quickly toward read speed. So diminishing returns can't be far over that. Quick runs at 16KB and no better, perhaps worse on average.
    This example also provides info on latency. It is written by the author of SdFat so probably example of best use and measurement.

    With a 64 MB SanDisk Ultra A1 class card I got the following showing write speed 10-17 MB/sec using the default: const size_t BUF_SIZE = 512;
    Code:
    Use a freshly formatted SD for best performance.
    
    Type any character to start
    FreeStack: 449064
    Type is exFAT
    Card size: 63.86 GB (GB = 1E9 bytes)
    
    Manufacturer ID: 0X3
    OEM ID: SD
    Product: SC64G
    Version: 8.0
    Serial number: 0X68B2B53B
    Manufacturing date: 6/2013
    
    FILE_SIZE_MB = 5
    BUF_SIZE = 512 bytes
    Starting write test, please wait.
    
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    12754.29,61371,22,28
    10351.30,111859,22,40
    
    ...
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    10459.58,62663,22,34
    10416.00,100311,22,37
    
    ...
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    17122.19,24594,22,24
    13623.11,41118,22,26
    
    Starting read test, please wait.
    
    read speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    22725.82,23,22,22
    22622.99,23,22,22
    Bumping that to: const size_t BUF_SIZE = 4096;
    Code:
    FreeStack: 445480
    Type is exFAT
    Card size: 63.86 GB (GB = 1E9 bytes)
    
    Manufacturer ID: 0X3
    OEM ID: SD
    Product: SC64G
    Version: 8.0
    Serial number: 0X68B2B53B
    Manufacturing date: 6/2013
    
    FILE_SIZE_MB = 5
    BUF_SIZE = 4096 bytes
    Starting write test, please wait.
    
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    15328.59,51667,179,222
    16384.00,53784,179,234
    
    ...
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    19596.55,24796,179,200
    13150.32,51250,179,222
    
    ...
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    19293.90,34573,179,209
    22308.57,3576,179,182
    
    Starting read test, please wait.
    
    read speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    22922.57,179,178,178
    22922.57,179,178,178

  9. #9
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    27,395
    This question has come up several times this year.

    Maybe we should add an example to the SD library to demonstrate high speed writing with use of a large buffer?

  10. #10
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,708
    Quote Originally Posted by PaulStoffregen View Post
    This question has come up several times this year.

    Maybe we should add an example to the SD library to demonstrate high speed writing with use of a large buffer?
    makes sense - was surprised the only one of the sort found was in the SdFat subsection.

  11. #11
    I never found that sort of data very useful. I prefer graphs.

    For my testing I wrote the current value from a hardware timer into the write buffer before issuing the write command. Then read the data back and process to find the time since the previous write command. Producing graphs like these. (Teensy 3.6 with no file system)

    Click image for larger version. 

Name:	single.gif 
Views:	12 
Size:	10.7 KB 
ID:	29861
    Click image for larger version. 

Name:	multi4.gif 
Views:	13 
Size:	9.7 KB 
ID:	29862

  12. #12
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    406
    I posted some example code comparing write speeds for the SDIO interface (on the T4.1) and the SPI interface (on the Teensy MicroMod) in another forum:

    https://forum.pjrc.com/threads/71482...l=1#post315781

    The takeaway is that the T4.1 can keep up with the OP's ADC as long as the buffer is more than 4KB. The buffer size requirement is dictated by the occasional long write time when the SD card has to erase a large storage block.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •