Increasing Sample Rate and SD Card Upload Rate (Teensy 4.1)

Status
Not open for further replies.

David1983

Member
Hi,

I'm working on a data collection device and am using the teensy 4.1 for its fast performance rate. However, while working on my project I have hit two road blocks:
1) The accelerometer I have been provided (Adafruit MMA8451) seems to cap at a sampling rate of 800Hz (from experiment and from the data sheet), however ideally I would want a sampling rate of 1.67kHz.
2) The SD card seems to cap at an upload rate of 500Hz, and I'm not sure how I would manage to be able to upload all the data I'm capturing then.

Are there any workarounds for this? I'm pretty new to microcontrollers as a whole so any advice would be appreciated. I've looked a bit into data buffers and SPI commands, but I'm still not completely sure on whether or not they could help improve my upload rates.

The code I have so far is provided below, but it's mostly just set up and functions given that I've mainly just been gathering data on the performance of the device. Any advice would be greatly appreciated!:D



Code:
#include <Adafruit_MAX31856.h>
#include <Wire.h>
#include <Adafruit_MMA8451.h>
#include <Adafruit_Sensor.h>
#include <SD.h>

#define DRDY_PIN 5

// Use software SPI: CS, DI, DO, CLK
//Adafruit_MAX31856 maxthermo = Adafruit_MAX31856(10, 11, 12, 13);
// use hardware SPI, just pass in the CS pin
Adafruit_MAX31856 maxthermo = Adafruit_MAX31856(10);
Adafruit_MMA8451 mma = Adafruit_MMA8451();

// GLOBAL VARIABLES ****************
const int chipSelect = BUILTIN_SDCARD;
const int arraySize = 32;

float temp_buffer[arraySize] = {1};
float accel_buffer[arraySize] = {1};
float freq_buffer[arraySize] = {1};

const char filename[] = "data.csv";
File dataFile;
//**********************************
//##################################
//**********TEENSY SETUP************
void setup() {
  // put your setup code here, to run once:
  
  Serial.begin(115200);
  while (!Serial) delay(10);
  
  Serial.println("Testing MAX31856 and MMA8451");
//********************************* MMA8451 SETUP **************************************************************************
  if (! mma.begin()) {
    Serial.println("MMA8451 couldnt start");
    while (1);
  }
  
  Serial.println("MMA8451 found!");

  mma.setRange(MMA8451_RANGE_2_G);
  Serial.print("Range = "); Serial.print(2 << mma.getRange());  
  Serial.println("G");
//**************************************************************************************************************************
//##########################################################################################################################
//******************************* MAX31856 SETUP ***************************************************************************
  if (!maxthermo.begin()) {
    Serial.println("Could not initialize thermocouple.");
    while (1) delay(10);
  }

  maxthermo.setThermocoupleType(MAX31856_TCTYPE_K);
  Serial.print("Thermocouple type: ");
  switch (maxthermo.getThermocoupleType() ) {
    case MAX31856_TCTYPE_B: Serial.println("B Type"); break;
    case MAX31856_TCTYPE_E: Serial.println("E Type"); break;
    case MAX31856_TCTYPE_J: Serial.println("J Type"); break;
    case MAX31856_TCTYPE_K: Serial.println("K Type"); break;
    case MAX31856_TCTYPE_N: Serial.println("N Type"); break;
    case MAX31856_TCTYPE_R: Serial.println("R Type"); break;
    case MAX31856_TCTYPE_S: Serial.println("S Type"); break;
    case MAX31856_TCTYPE_T: Serial.println("T Type"); break;
    case MAX31856_VMODE_G8: Serial.println("Voltage x8 Gain mode"); break;
    case MAX31856_VMODE_G32: Serial.println("Voltage x8 Gain mode"); break;
    default: Serial.println("Unknown"); break;
  }
  maxthermo.setConversionMode(MAX31856_CONTINUOUS);
  /*
  while(maxthermo.readThermocoupleTemperature() == 0){
    Serial.println("Working on retrieving proper data...");
    delay(100);
  }
  */
//*************************************************************************************************************************
//#########################################################################################################################
//******************************SD CARD INITIALIZATION*********************************************************************
  // init the SD card
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    // don't do anything more:
    while (1);
  }

  else {
    Serial.println("Found the card!");
  }

  // If you want to start from an empty file,
  // uncomment the next line:
  //SD.remove(filename);

  // try to open the file for writing
  dataFile = SD.open(filename, O_CREAT | O_WRITE);
  if (!dataFile) {
    Serial.print("error opening ");
    Serial.println(filename);
    while (1);
  }
//*************************************************************************************************************************
}

void loop() {
  /*fillBuffers();
  writeToFile();
  float temp_buffer[arraySize] = {-280};
  float accel_buffer[arraySize] = {-1};
  float freq_buffer[arraySize] = {-1};*/

  int beforeTime = micros();
  writeToFile();
  //float dataCheck = accelVal();
  //Serial.println(dataCheck);
  int afterTime = micros();
  int timeDiff = abs(afterTime - beforeTime);
  //Serial.println("time diff: " + String(timeDiff));
}

void fillBuffers(){
  for(int i = 0; i < arraySize; i++){
    temp_buffer[i] = tempVal();
    accel_buffer[i] = accelVal();
  }
}

float accelVal() {
  /* Get a new sensor event */ 
  sensors_event_t event; 
  mma.getEvent(&event);
  float x_accel = event.acceleration.x, y_accel = event.acceleration.y, z_accel = event.acceleration.z;
  /* Display the results (acceleration is measured in m/s^2) */
  /*
  Serial.print("X: \t"); Serial.print(event.acceleration.x); Serial.print("\t");
  Serial.print("Y: \t"); Serial.print(event.acceleration.y); Serial.print("\t");
  Serial.print("Z: \t"); Serial.print(event.acceleration.z); Serial.print("\t");
  Serial.println("m/s^2 ");
  */
  /*  
  // Read the 'raw' data in 14-bit counts
  mma.read();
  Serial.print("X:\t"); Serial.print(mma.x); 
  Serial.print("\tY:\t"); Serial.print(mma.y); 
  Serial.print("\tZ:\t"); Serial.print(mma.z); 
  Serial.println();
  */
  float accelScalar = sqrt(pow(x_accel, 2) + pow(y_accel, 2) + pow(z_accel, 2));
  return accelScalar;
}

float tempVal(){
  int count = 0;
  while (digitalRead(DRDY_PIN)) {
    if (count++ > 200) {
      count = 0;
      //Serial.print(".");
    }
  }
  
  //Serial.println(maxthermo.readThermocoupleTemperature());
  return float(maxthermo.readThermocoupleTemperature());
}

String formatData(float accelerationValue, float temperatureValue){
  String accel = String(accelerationValue);
  String temperature = String(temperatureValue);
  String dataString = accel + "," + temperature;
  return dataString;
}

void writeToFile(){
  for(int i = 0; i < arraySize; i++){
    int beforeTime = micros();
    String dataString = formatData(temp_buffer[i], accel_buffer[i]);
    dataFile.println(dataString);
    dataFile.flush();
    int afterTime = micros();
    int timeDiff = afterTime - beforeTime;
    Serial.println(timeDiff);
  }
}
 
You are flushing every sample to SD card, flushing may involve several segment writes, segments are 512 bytes each.

Better to flush regularly in time, such as every second, than every sample.
 
First, upgrade to 1.54-beta7, if you haven't already. The old slow SD lib is gone, now just a wrapper for fast SdFat.

You may need to use IntervalTimer to run your code that collects incoming data, so it can interrupt while the SD card is busy writing. Obviously you need to pile up that incoming data into a big buffer and then later store it all as a single large SD write.

You might also consider whether higher sample rate is even worthwhile. Most MEMS sensors have pretty limited bandwidth, because they're demodulating a high frequency and then low pass filtering the modulator output to create a baseband signal.
 
I've been trying to update to 1.54 beta but I can't seem to find it in the list of installable versions on the arduino IDE. I tried searching online too and downloaded teensyduino 1.54 beta but it still says version 1.24 is the most recent version of the SD library. As for the higher sample rate, I believe my work wants it because they need very frequent measurements to accurately extrapolate the data they want from it. If it's possible to make the sample rate higher, I think that'd be desirable, tough it's also fine if it's just impractical.
 
Haha that's actually the link I used, I'm on windows so I just clicked the windows link there at the top and followed the download procedure. Did I have to uninstall my old teensyduino before or something? Sorry if these are simple questions.
 
After downloading, you do need to actually run the installer. Quit Arduino before you run it.

If using Windows with Arduino installed the default way (the EXE installer, not the Windows store app) just click Next in the installer until it's done.

When you run Arduino again, click Help > About to check which version is installed.
 
Oh ok help says 1.54 beta 4 is installed, I notice you have a note about installing on a fresh arduino if I had this version previously. Should I uninstall and reinstall arduino then?
 
Perfect, now I'm having issues locating the SD card though. It keeps saying the card can't be located, but also BUILTIN_SDCARD isn't recognized. Is there a new variable that's used?
 
Wait nevermind it works, but it actually seems to be slower than before... I was clocking at about 500Hz for uploading, and now I'm only reaching ~330Hz. Delaying the flushes doesn't seem to work either, as it just uploads a 0 whenever the flush doesn't occur.
 
Wait nevermind it works, but it actually seems to be slower than before... I was clocking at about 500Hz for uploading, and now I'm only reaching ~330Hz. Delaying the flushes doesn't seem to work either, as it just uploads a 0 whenever the flush doesn't occur.

You will never get high speed low latency results with flush()/sync(). flush() is just a call to sync().
Code:
  /** Arduino name for sync() */
  void flush() {sync();}

sync() causes a huge amount of extra I/O. The data buffers are written to the SD, the directory entry is read, updated and written. Each data sector on the SD will be rewritten a number of times. SD cards have huge internal buffers and flash pages so each sync() call may cause a number of 32KB flash pages to be rewritten to a new flash page and the old pages are part of 512 KB Record Units which must be erased. You are increasing the internal I/O in the card by a huge factor.

I recently added several features to SdFat for simple high rate data logging. This example uses no DMA or ISRs but can log at about 25 kHz on a Teensy 4.1.

This example uses a special ring buffer that allows you to print text into the ring buffer, no binary to convert.
See this section:
Code:
    // Print spareMicros into the RingBuf as test data.
    rb.print(spareMicros);
    rb.write(',');
    // Print adc into RingBuf.
    rb.println(adc);

You can just replace the above with:
Code:
    rb.println(dataString);
 
Yeah I read in other forums that calling flush() that often would slow down my code, I just couldn't find a way around it. Thanks for providing that solution, I'll be sure to check it out! Do I just copy the set up code from the sdiologger? Or should I parse through and take what I think I need?
 
Yeah I read in other forums that calling flush() that often would slow down my code, I just couldn't find a way around it. Thanks for providing that solution, I'll be sure to check it out! Do I just copy the set up code from the sdiologger? Or should I parse through and take what I think I need?

Sometimes I find users call flush/sync since they do not properly close a file. close() just calls sync() then marks the file closed.
Code:
bool ExFatFile::close() {
  bool rtn = sync();
  m_attributes = FILE_ATTR_CLOSED;
  m_flags = 0;
  return rtn;
}

With any example you should run it to understand what is does then take the parts you need and modify for your use.

The key ideas that make this logger fast and avoid missing samples are:

Pre-allocate a large contiguous file then trim the unused portion - avoids overhead for search and addition of clusters.

Use a large ring buffer so time critical print/write calls are not blocked when the SD is busy. A simple way to separate file write from sensor read.

file.isBusy() avoids stalling on a write while the SD is busy with wear leveling, erasing flash, programming flash, copying partial flash pages to garbage collect...

rb.writeOut(512) in the following code only takes about 5 μs independent of the SD card. It just sends 512 bytes to the SD controller FIFO and the data is sent to the SD while program execution continues.

Code:
    if (n >= 512 && !file.isBusy()) {
      // Not busy only allows one sector before possible busy wait.
      // Write one sector from RingBuf to file.
      if (512 != rb.writeOut(512)) {
        Serial.println("writeOut failed");
        break;
      }
    }

Writing only 512 bytes per write call gives you an effective 100MB/sec write rate. Much better than a large write which will go at a max speed of about 22 MB/sec so it save CPU time for other tasks.
 
Last edited:
Status
Not open for further replies.
Back
Top