Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 9 of 9

Thread: SdFat DMA over-clocked 3 Msps ADC logger

  1. #1
    Senior Member
    Join Date
    Nov 2012
    Posts
    321

    SdFat DMA over-clocked 3 Msps ADC logger

    I have been working on new features for SdFat. Two are useful for Teensy 3.6 and Teensy 4.1.

    The first is a ring buffer that is tightly integrated with SdFat and can be called from an ISR. As a test, I did a DMA ADC logger and stress tested it with the max ADC over-clocking on Teensy 3.6. Here is a chart of a 50 kHz square wave at 3 million samples per second.

    Click image for larger version. 

Name:	TeensySqr.jpg 
Views:	54 
Size:	71.4 KB 
ID:	23303

    I have attached the the .ino file for the logger and RingBuf.h, the template for the ring buffer. You can run them with SdFat 2.0.4.

    The second feature is a rework of the Teensy SDIO driver. I have been able to get sd busy to work. I also overlap sending data to the sd with execution of the program. I do this by writing a sector to the SDHC FIFO then returning while the data is sent over the SDIO bus. On Teensy 4.1, writing the first sector takes 10-11 usec which includes time to put the controller in write mode. Subsequent sectors take 5 usec.

    the isBusy feature plus the short write time allows a simple non DMA program to log data at over 25 kHz.

    Click image for larger version. 

Name:	Teensy50kHz.jpg 
Views:	56 
Size:	70.8 KB 
ID:	23306

    Here is the loop that logged the data for the above chart. It logged by printing text to a RingBuf object.

    Code:
    // 400 sector ring buffer for FsFile type.
    RingBuf<FsFile, 400*512> rb;
    
      // Initialize the RingBuf - use a large pre-allocated exFAT file.
      rb.begin(&file); 
      
      // Used for controlling the log interval.
      uint32_t logTime = micros();
      while (!Serial.available()) {
        // Write data to SD if a sector is in the RingBuf.   
        if (rb.bytesUsed() >= 512 && !file.isBusy()) {
          // Writing full sectors assures no copy to SdFat internal cache buffer.
          // Write of first sector takes about 11 usec, 5 usec each for remainder.
          if (512 != rb.writeOut(512)) {
            // Handle SD write error here.
          }
        }
        // Time for next point - log at 25 kHz.
        logTime += 40;
        // Spare time before next point - negative if time overrun.
        int32_t spareMicros = logTime - micros();
        // Wait until time to log data.
        while (micros() < logTime) {}    
        
        // Read ADC0 this takes about 17 usec on Teensy 4.
        uint16_t adc = analogRead(0);
        // Print spareMicros into the RingBuf as test data.
        rb.print(spareMicros);
        rb.write(',');
        // Print adc into RingBuf.
        rb.println(adc);
        if (rb.getWriteError()) {
          // Handle error caused by too few free bytes in RingBuf.
        }
      }
    I will release a beta of the new driver soon. The isBusy feature is of limited use for Teensy 3.6 since it can't do infinite writes before a reset, only 0xffff sectors.
    Attached Files Attached Files

  2. #2
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    563
    Quote Originally Posted by Bill Greiman View Post
    I have been working on new features for SdFat. Two are useful for Teensy 3.6 and Teensy 4.1.

    The first is a ring buffer that is tightly integrated with SdFat and can be called from an ISR. As a test, I did a DMA ADC logger and stress tested it with the max ADC over-clocking on Teensy 3.6. Here is a chart of a 50 kHz square wave at 3 million samples per second.

    Click image for larger version. 

Name:	TeensySqr.jpg 
Views:	54 
Size:	71.4 KB 
ID:	23303

    I have attached the the .ino file for the logger and RingBuf.h, the template for the ring buffer. You can run them with SdFat 2.0.4.

    The second feature is a rework of the Teensy SDIO driver. I have been able to get sd busy to work. I also overlap sending data to the sd with execution of the program. I do this by writing a sector to the SDHC FIFO then returning while the data is sent over the SDIO bus. On Teensy 4.1, writing the first sector takes 10-11 usec which includes time to put the controller in write mode. Subsequent sectors take 5 usec.

    the isBusy feature plus the short write time allows a simple non DMA program to log data at over 25 kHz.

    Click image for larger version. 

Name:	Teensy50kHz.jpg 
Views:	56 
Size:	70.8 KB 
ID:	23306

    Here is the loop that logged the data for the above chart. It logged by printing text to a RingBuf object.

    Code:
    // 400 sector ring buffer for FsFile type.
    RingBuf<FsFile, 400*512> rb;
    
      // Initialize the RingBuf - use a large pre-allocated exFAT file.
      rb.begin(&file); 
      
      // Used for controlling the log interval.
      uint32_t logTime = micros();
      while (!Serial.available()) {
        // Write data to SD if a sector is in the RingBuf.   
        if (rb.bytesUsed() >= 512 && !file.isBusy()) {
          // Writing full sectors assures no copy to SdFat internal cache buffer.
          // Write of first sector takes about 11 usec, 5 usec each for remainder.
          if (512 != rb.writeOut(512)) {
            // Handle SD write error here.
          }
        }
        // Time for next point - log at 25 kHz.
        logTime += 40;
        // Spare time before next point - negative if time overrun.
        int32_t spareMicros = logTime - micros();
        // Wait until time to log data.
        while (micros() < logTime) {}    
        
        // Read ADC0 this takes about 17 usec on Teensy 4.
        uint16_t adc = analogRead(0);
        // Print spareMicros into the RingBuf as test data.
        rb.print(spareMicros);
        rb.write(',');
        // Print adc into RingBuf.
        rb.println(adc);
        if (rb.getWriteError()) {
          // Handle error caused by too few free bytes in RingBuf.
        }
      }
    I will release a beta of the new driver soon. The isBusy feature is of limited use for Teensy 3.6 since it can't do infinite writes before a reset, only 0xffff sectors.
    HI Bill,

    I am really anxious to try this out with the two versions of SdFat that I am working on for MSC. I have two versions of SdFat that I have modified adding my USB Host Mass Storage driver to them as a block device. One of the most modified files are 'SdFat.h'. This version of SdFat is more of what I call an inline version. It is the most compatible version to us with @Paul's 'SD.h' and 'FS.h' file abstraction libraries. The second version of SdFat is less invasive to your Sdfat library and parallels 'SD.h' and 'SdFat.h'. Both version are working great with SdFat. The block device driver in the src folder is 'USBmsController'.

    Here is the thread with the links to the libraries:
    https://forum.pjrc.com/threads/64784...ght=sdfat+ways

    I have to admit that with my skill level of c++, it has been challenging to understand and follow all of the work you have put into SdFat.
    I just figured out how to pull volume labels for both ExFat and Fat32.
    Using version 2.0.0 and I know I will have to use version 2.0.4 for use with the RingBuf.
    Thanks for a great library

  3. #3
    Senior Member
    Join Date
    Nov 2012
    Posts
    321
    I have an example using SdFat for USB mass storage with a Mega ADK. I once posted "UsbFat" on github but found almost no interest. I think it was too early and the ADK was never popular.

  4. #4
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    563
    Quote Originally Posted by Bill Greiman View Post
    I have an example using SdFat for USB mass storage with a Mega ADK. I once posted "UsbFat" on github but found almost no interest. I think it was too early and the ADK was never popular.
    You know Bill, all the times I have visited your GitHub repository I never took the time to check it all out. Was usually checking or getting SdFat.
    It's interesting that you called it UsbFat because that is what I named my version of SdFat.h in my version of SdFat-parallel. Will be checking it out

    Thanks.
    Last edited by wwatson; 01-19-2021 at 12:09 AM. Reason: Typo again

  5. #5
    Senior Member
    Join Date
    Nov 2012
    Posts
    321
    I have not updated UsbFat for six years. I have been meaning to add an example of USB storage to the current SdFat. This example shows how to use any block device with SdFat.

    The driver uses a USB shield library for a commonly available type of USB host shield.

    I intend to improve support for BlockDevice options. I would like to use SdFat unmodified and have add-on libraries for various devices. I have used a big hard drive, externally powered.

    Code:
    // Edit SdFatConfig.h and enable generic block devices.
    // #define USE_BLOCK_DEVICE_INTERFACE 1
    #include "UsbMscDriver.h"
    
    USB usb;
    BulkOnly bulk(&usb);
    UsbMscDriver usbKey(&bulk);
    
    //FatVolume key;
    //FatFile file;
    
    FsVolume key;
    FsFile file;
    
    //uint8_t lun;
    
    void setup() {
      Serial.begin(9600);
      while (!Serial) {}
      Serial.println(F("Type any character to start"));
      while (!Serial.available()) {}
     
      if (!initUSB(&usb)) {
        Serial.println("initUSB failed");
        while(1){}
      }
      
      if (!key.begin(&usbKey)) {
        Serial.println("key.begin failed");
        while(1) {}
      }
      if (!file.open("usbtest.txt", FILE_WRITE)) {
        Serial.println("file.open failed");
        while(1) {}    
      }
      file.println("test line");
      file.close();
      key.ls(LS_DATE | LS_SIZE);
    }
    
    void loop() {
    }
    Here is the wrapper for the USB driver:


    UsbMscDriver.h
    Code:
    #ifndef UsbMscDriver_h
    #define UsbMscDriver_h
    #include "SdFat.h"
    
    // If the next include fails, install the USB_HOST_SHIELD library.
    #include <masstorage.h>
    //------------------------------------------------------------------------------
    /** Simple USB init function.
     * \param[in] usb USB host object to be initialized.
     * \return true for success or false for failure.
     */
    bool initUSB(USB* usb);
    //------------------------------------------------------------------------------
    /** Print debug messages if USB_FAT_DBG_MODE is nonzero */
    #define USB_FAT_DBG_MODE 1
    //------------------------------------------------------------------------------
    /** Maximum time to initialize the USB bus */
    #define TIMEOUT_MILLIS 400000L
    class UsbMscDriver : public BlockDeviceInterface {
     public:
      UsbMscDriver(BulkOnly* bulk)  : m_lun(0), m_bulk(bulk) {}
      bool readSector(uint32_t sector, uint8_t* dst) {
        uint8_t rc = m_bulk->Read(m_lun, sector, 512, 1, dst);
        Serial.print("rc ");Serial.println(rc, HEX);
        return 0 == rc;
      }
      uint32_t sectorCount() {
        return m_bulk->GetCapacity(m_lun);
      }
      bool syncDevice() {return true;}
      bool writeSector(uint32_t sector, const uint8_t* src) {
        return  0 == m_bulk->Write(m_lun, sector, 512, 1, src);    
      }
     
      bool readSectors(uint32_t sector, uint8_t* dst, size_t ns) {
        return 0 == m_bulk->Read(m_lun, sector, 512, ns, dst);   
      }
      bool writeSectors(uint32_t sector, const uint8_t* src, size_t ns) {
        return  0 == m_bulk->Write(m_lun, sector, 512, ns, src);      
      }
    
     private:
      uint8_t m_lun;
      BulkOnly* m_bulk;   
    };
    #endif  // UsbMscDriver_h
    UsbMscDriver..cpp
    Code:
    #include "UsbMscDriver.h"
    //------------------------------------------------------------------------------
    bool initUSB(USB* usb) {
     uint8_t last_state = 0;
     uint8_t current_state = 0;
      uint32_t m = millis();
      for (uint8_t i = 0; usb->Init(1000) == -1; i++) {
        if (USB_FAT_DBG_MODE) {
          Serial.println(F("No USB HOST Shield?"));
        }
        if (i > 10) {
          return false;
        }
      }
    #if USB_FAT_DBG_MODE
      Serial.print(F("Host initialized, ms: "));
      Serial.println(millis() - m);
    #endif  // USB_FAT_DBG_MODE
    
      usb->vbusPower(vbus_on);
    #if USB_FAT_DBG_MODE  
      Serial.print(F("USB powered, ms: "));
      Serial.println(millis() - m);
    #endif  // USB_FAT_DBG_MODE
      
      while ((millis() - m) < TIMEOUT_MILLIS) {
        usb->Task();  
        current_state = usb->getUsbTaskState();
    #if USB_FAT_DBG_MODE    
        if (last_state != current_state) {
          Serial.print(F("USB state: "));
          Serial.print(current_state, HEX);
          Serial.print(F(", ms: "));
          Serial.println(millis() - m);
        }
        last_state = current_state;
    #endif  // USB_FAT_DBG_MODE    
        if(current_state == USB_STATE_RUNNING) {
          return true;
        }
      }
      return false;
    }

  6. #6
    Senior Member
    Join Date
    Nov 2012
    Posts
    321
    I posted SdFat-beta with the new Teensy SDIO driver and two examples that demonstrate fast data logging using the new driver.

    Try the TeensyDmaAdcLogger and TeensySdioLogger examples.

  7. #7
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    563
    @Bill - Thanks for providing this. It gives me direction for the best way to include Teensy USBHost Mass Storage devices with SdFat. I'm sure @Paul has his plans as well. This weekend I will use this as template to add my driver as a block device to SdFat. The nice thing about the EHCI USB interface on the T3.6 and T4.x is that USBHost_t36 takes care of pipe stalls and such automatically thereby simplifying error codes and recovery.

  8. #8
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    563
    @Bill - Thanks to the examples you provided I now have my MSC Mass Storage driver completely isolated from an unmodified SdFat. Everything is working as it was before but with just one include of 'SdFat.h'. I still have a lot of work to do on it but at least I have good start. It now resembles littleFS in structure.
    Want to test with your latest version of SdFat-beta

    Again thanks for taking the time to help.

  9. #9
    Hello,

    Amazing potential for sampling at 3 MSPS. Has anyone got this running on a T4.1, I keep getting the following error:

    FreeStack: 240928
    error: file.preAllocate failed.

    If I try and run this on a T3.6 then the error is:

    FreeStack: 50296
    error: file.preAllocate failed

    Run in VSCode and arduino IDE, double checked Latest SDFAT library installed, still no luck. I realise that I am probably doing something dumb but any hints as to what that may be will be appreciated.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •