SdFat DMA over-clocked 3 Msps ADC logger

Bill Greiman

Well-known member
I have been working on new features for SdFat. Two are useful for Teensy 3.6 and Teensy 4.1.

The first is a ring buffer that is tightly integrated with SdFat and can be called from an ISR. As a test, I did a DMA ADC logger and stress tested it with the max ADC over-clocking on Teensy 3.6. Here is a chart of a 50 kHz square wave at 3 million samples per second.

TeensySqr.jpg

I have attached the the .ino file for the logger and RingBuf.h, the template for the ring buffer. You can run them with SdFat 2.0.4.

The second feature is a rework of the Teensy SDIO driver. I have been able to get sd busy to work. I also overlap sending data to the sd with execution of the program. I do this by writing a sector to the SDHC FIFO then returning while the data is sent over the SDIO bus. On Teensy 4.1, writing the first sector takes 10-11 usec which includes time to put the controller in write mode. Subsequent sectors take 5 usec.

the isBusy feature plus the short write time allows a simple non DMA program to log data at over 25 kHz.

Teensy50kHz.jpg

Here is the loop that logged the data for the above chart. It logged by printing text to a RingBuf object.

Code:
// 400 sector ring buffer for FsFile type.
RingBuf<FsFile, 400*512> rb;

  // Initialize the RingBuf - use a large pre-allocated exFAT file.
  rb.begin(&file); 
  
  // Used for controlling the log interval.
  uint32_t logTime = micros();
  while (!Serial.available()) {
    // Write data to SD if a sector is in the RingBuf.   
    if (rb.bytesUsed() >= 512 && !file.isBusy()) {
      // Writing full sectors assures no copy to SdFat internal cache buffer.
      // Write of first sector takes about 11 usec, 5 usec each for remainder.
      if (512 != rb.writeOut(512)) {
        // Handle SD write error here.
      }
    }
    // Time for next point - log at 25 kHz.
    logTime += 40;
    // Spare time before next point - negative if time overrun.
    int32_t spareMicros = logTime - micros();
    // Wait until time to log data.
    while (micros() < logTime) {}    
    
    // Read ADC0 this takes about 17 usec on Teensy 4.
    uint16_t adc = analogRead(0);
    // Print spareMicros into the RingBuf as test data.
    rb.print(spareMicros);
    rb.write(',');
    // Print adc into RingBuf.
    rb.println(adc);
    if (rb.getWriteError()) {
      // Handle error caused by too few free bytes in RingBuf.
    }
  }
I will release a beta of the new driver soon. The isBusy feature is of limited use for Teensy 3.6 since it can't do infinite writes before a reset, only 0xffff sectors.
 

Attachments

  • RingBuf.h
    9.6 KB · Views: 82
  • TeensyDmaAdc.ino
    6.9 KB · Views: 89
I have been working on new features for SdFat. Two are useful for Teensy 3.6 and Teensy 4.1.

The first is a ring buffer that is tightly integrated with SdFat and can be called from an ISR. As a test, I did a DMA ADC logger and stress tested it with the max ADC over-clocking on Teensy 3.6. Here is a chart of a 50 kHz square wave at 3 million samples per second.

View attachment 23303

I have attached the the .ino file for the logger and RingBuf.h, the template for the ring buffer. You can run them with SdFat 2.0.4.

The second feature is a rework of the Teensy SDIO driver. I have been able to get sd busy to work. I also overlap sending data to the sd with execution of the program. I do this by writing a sector to the SDHC FIFO then returning while the data is sent over the SDIO bus. On Teensy 4.1, writing the first sector takes 10-11 usec which includes time to put the controller in write mode. Subsequent sectors take 5 usec.

the isBusy feature plus the short write time allows a simple non DMA program to log data at over 25 kHz.

View attachment 23306

Here is the loop that logged the data for the above chart. It logged by printing text to a RingBuf object.

Code:
// 400 sector ring buffer for FsFile type.
RingBuf<FsFile, 400*512> rb;

  // Initialize the RingBuf - use a large pre-allocated exFAT file.
  rb.begin(&file); 
  
  // Used for controlling the log interval.
  uint32_t logTime = micros();
  while (!Serial.available()) {
    // Write data to SD if a sector is in the RingBuf.   
    if (rb.bytesUsed() >= 512 && !file.isBusy()) {
      // Writing full sectors assures no copy to SdFat internal cache buffer.
      // Write of first sector takes about 11 usec, 5 usec each for remainder.
      if (512 != rb.writeOut(512)) {
        // Handle SD write error here.
      }
    }
    // Time for next point - log at 25 kHz.
    logTime += 40;
    // Spare time before next point - negative if time overrun.
    int32_t spareMicros = logTime - micros();
    // Wait until time to log data.
    while (micros() < logTime) {}    
    
    // Read ADC0 this takes about 17 usec on Teensy 4.
    uint16_t adc = analogRead(0);
    // Print spareMicros into the RingBuf as test data.
    rb.print(spareMicros);
    rb.write(',');
    // Print adc into RingBuf.
    rb.println(adc);
    if (rb.getWriteError()) {
      // Handle error caused by too few free bytes in RingBuf.
    }
  }
I will release a beta of the new driver soon. The isBusy feature is of limited use for Teensy 3.6 since it can't do infinite writes before a reset, only 0xffff sectors.

HI Bill,

I am really anxious to try this out with the two versions of SdFat that I am working on for MSC. I have two versions of SdFat that I have modified adding my USB Host Mass Storage driver to them as a block device. One of the most modified files are 'SdFat.h'. This version of SdFat is more of what I call an inline version. It is the most compatible version to us with @Paul's 'SD.h' and 'FS.h' file abstraction libraries. The second version of SdFat is less invasive to your Sdfat library and parallels 'SD.h' and 'SdFat.h'. Both version are working great with SdFat. The block device driver in the src folder is 'USBmsController'.

Here is the thread with the links to the libraries:
https://forum.pjrc.com/threads/64784-Sdfat-beta-now-ported-to-USB-Mass-Storage-devices-two-ways?highlight=sdfat+ways

I have to admit that with my skill level of c++, it has been challenging to understand and follow all of the work you have put into SdFat.
I just figured out how to pull volume labels for both ExFat and Fat32.
Using version 2.0.0 and I know I will have to use version 2.0.4 for use with the RingBuf.
Thanks for a great library:)
 
I have an example using SdFat for USB mass storage with a Mega ADK. I once posted "UsbFat" on github but found almost no interest. I think it was too early and the ADK was never popular.
 
I have an example using SdFat for USB mass storage with a Mega ADK. I once posted "UsbFat" on github but found almost no interest. I think it was too early and the ADK was never popular.

You know Bill, all the times I have visited your GitHub repository I never took the time to check it all out. Was usually checking or getting SdFat.
It's interesting that you called it UsbFat because that is what I named my version of SdFat.h in my version of SdFat-parallel. Will be checking it out:)

Thanks.
 
Last edited:
I have not updated UsbFat for six years. I have been meaning to add an example of USB storage to the current SdFat. This example shows how to use any block device with SdFat.

The driver uses a USB shield library for a commonly available type of USB host shield.

I intend to improve support for BlockDevice options. I would like to use SdFat unmodified and have add-on libraries for various devices. I have used a big hard drive, externally powered.

Code:
// Edit SdFatConfig.h and enable generic block devices.
// #define USE_BLOCK_DEVICE_INTERFACE 1
#include "UsbMscDriver.h"

USB usb;
BulkOnly bulk(&usb);
UsbMscDriver usbKey(&bulk);

//FatVolume key;
//FatFile file;

FsVolume key;
FsFile file;

//uint8_t lun;

void setup() {
  Serial.begin(9600);
  while (!Serial) {}
  Serial.println(F("Type any character to start"));
  while (!Serial.available()) {}
 
  if (!initUSB(&usb)) {
    Serial.println("initUSB failed");
    while(1){}
  }
  
  if (!key.begin(&usbKey)) {
    Serial.println("key.begin failed");
    while(1) {}
  }
  if (!file.open("usbtest.txt", FILE_WRITE)) {
    Serial.println("file.open failed");
    while(1) {}    
  }
  file.println("test line");
  file.close();
  key.ls(LS_DATE | LS_SIZE);
}

void loop() {
}
Here is the wrapper for the USB driver:


UsbMscDriver.h
Code:
#ifndef UsbMscDriver_h
#define UsbMscDriver_h
#include "SdFat.h"

// If the next include fails, install the USB_HOST_SHIELD library.
#include <masstorage.h>
//------------------------------------------------------------------------------
/** Simple USB init function.
 * \param[in] usb USB host object to be initialized.
 * \return true for success or false for failure.
 */
bool initUSB(USB* usb);
//------------------------------------------------------------------------------
/** Print debug messages if USB_FAT_DBG_MODE is nonzero */
#define USB_FAT_DBG_MODE 1
//------------------------------------------------------------------------------
/** Maximum time to initialize the USB bus */
#define TIMEOUT_MILLIS 400000L
class UsbMscDriver : public BlockDeviceInterface {
 public:
  UsbMscDriver(BulkOnly* bulk)  : m_lun(0), m_bulk(bulk) {}
  bool readSector(uint32_t sector, uint8_t* dst) {
    uint8_t rc = m_bulk->Read(m_lun, sector, 512, 1, dst);
    Serial.print("rc ");Serial.println(rc, HEX);
    return 0 == rc;
  }
  uint32_t sectorCount() {
    return m_bulk->GetCapacity(m_lun);
  }
  bool syncDevice() {return true;}
  bool writeSector(uint32_t sector, const uint8_t* src) {
    return  0 == m_bulk->Write(m_lun, sector, 512, 1, src);    
  }
 
  bool readSectors(uint32_t sector, uint8_t* dst, size_t ns) {
    return 0 == m_bulk->Read(m_lun, sector, 512, ns, dst);   
  }
  bool writeSectors(uint32_t sector, const uint8_t* src, size_t ns) {
    return  0 == m_bulk->Write(m_lun, sector, 512, ns, src);      
  }

 private:
  uint8_t m_lun;
  BulkOnly* m_bulk;   
};
#endif  // UsbMscDriver_h

UsbMscDriver..cpp
Code:
#include "UsbMscDriver.h"
//------------------------------------------------------------------------------
bool initUSB(USB* usb) {
 uint8_t last_state = 0;
 uint8_t current_state = 0;
  uint32_t m = millis();
  for (uint8_t i = 0; usb->Init(1000) == -1; i++) {
    if (USB_FAT_DBG_MODE) {
      Serial.println(F("No USB HOST Shield?"));
    }
    if (i > 10) {
      return false;
    }
  }
#if USB_FAT_DBG_MODE
  Serial.print(F("Host initialized, ms: "));
  Serial.println(millis() - m);
#endif  // USB_FAT_DBG_MODE

  usb->vbusPower(vbus_on);
#if USB_FAT_DBG_MODE  
  Serial.print(F("USB powered, ms: "));
  Serial.println(millis() - m);
#endif  // USB_FAT_DBG_MODE
  
  while ((millis() - m) < TIMEOUT_MILLIS) {
    usb->Task();  
    current_state = usb->getUsbTaskState();
#if USB_FAT_DBG_MODE    
    if (last_state != current_state) {
      Serial.print(F("USB state: "));
      Serial.print(current_state, HEX);
      Serial.print(F(", ms: "));
      Serial.println(millis() - m);
    }
    last_state = current_state;
#endif  // USB_FAT_DBG_MODE    
    if(current_state == USB_STATE_RUNNING) {
      return true;
    }
  }
  return false;
}
 
@Bill - Thanks for providing this. It gives me direction for the best way to include Teensy USBHost Mass Storage devices with SdFat. I'm sure @Paul has his plans as well. This weekend I will use this as template to add my driver as a block device to SdFat. The nice thing about the EHCI USB interface on the T3.6 and T4.x is that USBHost_t36 takes care of pipe stalls and such automatically thereby simplifying error codes and recovery.
 
@Bill - Thanks to the examples you provided I now have my MSC Mass Storage driver completely isolated from an unmodified SdFat. Everything is working as it was before but with just one include of 'SdFat.h'. I still have a lot of work to do on it but at least I have good start. It now resembles littleFS in structure.
Want to test with your latest version of SdFat-beta:)

Again thanks for taking the time to help.
 
Hello,

Amazing potential for sampling at 3 MSPS. Has anyone got this running on a T4.1, I keep getting the following error:

FreeStack: 240928
error: file.preAllocate failed.

If I try and run this on a T3.6 then the error is:

FreeStack: 50296
error: file.preAllocate failed

Run in VSCode and arduino IDE, double checked Latest SDFAT library installed, still no luck. I realise that I am probably doing something dumb but any hints as to what that may be will be appreciated.
 
Back
Top