Stream Data from Serial1 to SD Card, Teensy 3.6

Status
Not open for further replies.

Don Kelly

Well-known member
I do a lot of GPS testing, and I'd like to just capture NMEA data (RMC messages) as it streams to the SD card on Teensy 3.6 Serial1. I have great success using the Sparkfun OpenLog SD card with T3.2s and T4.0s, but now I'm trying to get it working on the built-in T3.6 SD card. My routine below works fairly well at 1Hz on the T3.6, putting one RMC message per line. But when I bump the output to 10Hz, everything gets written to the a single line because the last couple of characters of each RMC message seem to not get captured.

Is there a library (or sketch) that just does what the OpenLog does, i.e. just simply captures the entire NMEA data cleanly as it streams to Serial?

Thanks for any advice!


Code:
// The Code...

#include <SPI.h>
#include <SD.h>

#define gpsPort Serial1

File GPS_File;
void setup() {

  Serial.begin(115200);
  gpsPort.begin(115200);

  delay(500);
  
  Serial.println("Am Here");

  if (!SD.begin(BUILTIN_SDCARD)) {
    Serial.println("Card failed, or not present");
    return;
  }
  Serial.println("Card initialized.");

  // Remove old file
  SD.remove("datalog.txt");

  // Create new file
  GPS_File = SD.open("datalog.txt", FILE_WRITE);
  GPS_File.close();
  
  //GPS_File.write("START\n");

  // Enable LED pin
  pinMode(ledPin, OUTPUT);

}

void loop() {
  if (gpsPort.available())
  {
    String Buffer = "";
    while (gpsPort.available())
    {
        char GPSRX = gpsPort.read();
        Buffer += GPSRX;

    }
     
    // Open the file
    GPS_File = SD.open("datalog.txt", FILE_WRITE);

    // If the file is available, write to it:
    if (GPS_File) {
      GPS_File.print(Buffer);
      GPS_File.close();
      // print to the serial port too:
      //Serial.print(Buffer);
    }  
    // If the file isn't open, pop up an error
    else {
      Serial.println("Error opening datalog.txt");
    } 
    
    
  }
}
 
Last edited by a moderator:
As a crude first step to let GPS data arrive perhaps:
Code:
    while (gpsPort.available())
    {
        [B]delay(1);[/B]
        char GPSRX = gpsPort.read();
        Buffer += GPSRX;
    }

If the .available() catches the string first bytes - it will exhaust those received before the message completes. The 1ms will probably trigger once - maybe 2-3 times in this case depending on message length and will waste time - but this is just to test all else is well for simple logging. It will assure getting a complete message before logging to SD. "SD.h" writes can take 10 to 100 or more ms to complete. There is a new thread with decent logging code that buffer incoming data for more efficient and fewer writes. And uses an _isr() to log the data - this should be safe for .read() just transferring buffered data as well - noted below. But the code above will overflow Serial1 buffer on long SD writes. There are observed times in the linked thread below.

If there is a terminator char on the incoming string keying on that would allow better response.

User added code near the current thread end tested here to work well : Teensy-3-6-SD-Card-missing-time-in-milliseconds { posts #6 and #10 }
There is an SD.h and SDFat.h implementation - SDFat is way faster to write. The dual buffers could read as you do byte by byte until one buffer is full - then mark that buffer to WRITE and switch to the other buffer as done in that code, and size the buffers in bocks of 512 bytes. That avoids file open close on each write as well. Then as in that code start a timer ( intervalTimer or the raw code there ) and pull out Serial1 data into the buffers. { current writes of 64KB SD buffers are taking 2.88 ms - up linearly from 0.77 ms when it was 16 KB } with rare spikes using SDFat and exFAT formatted flash card.
 
You aren't making sure that you've read the whole line before writing it.
Try replacing this:
Code:
    while (gpsPort.available())
    {
        char GPSRX = gpsPort.read();
        Buffer += GPSRX;

    }
with this:
Code:
    while(1) {
      while (gpsPort.available() < 1);
      char GPSRX = gpsPort.read();
      Buffer += GPSRX;
      if(GPSRX == '\n')break;
    }

Pete
 
Pete's post is code for this note in prior post : If there is a terminator char on the incoming string keying on that would allow better response.

But both will fail as noted over time with slow write speed of SD and Serial1 buffer getting over run when the write takes in excess of 100 ms at 10 Hz. Linked post and notes on T_4.1 Beta thread about speed observed today .
 
Pete,
Your trick works. Thx. Here's what I see now (on both the T3.6 and T4.1):

1. 10Hz NMEA RMC messages write to SD card just fine. I can also tested doing other tasks in the loop (blinking LED) and still saved the RMC messages just fine.
2. 20Hz NMEA RMC messages write to SD card just fine as well, if nothing else in the Void loop. If I add a small task, even just blinking LED, I get dropped characters at the end of the line.

Low-cost GPS receivers are moving towards 40Hz rates, but right now I can get by with the 10Hz rate for what I'm doing. But I have to say that it seems that a simple RMC message should be able to be recorded at 20+ Hz with no issues. I'll have to go see if the Sparkfun OpenLog SD can do 20 Hz, as I've used it routinely for 10 Hz.
 
You'll have to rewrite the loop function if you want to do more than just read the RMC message. The way it is written now, if there are any characters available from the GPS, it hangs until it has written a whole line to the SD.
Replace the entire loop() function with this:
Code:
String Buffer = "";
void loop() {
  while (gpsPort.available() >= 1)
  {
    char GPSRX = gpsPort.read();
    Buffer += GPSRX;
    if(GPSRX == '\n') {
      // Open the file
      GPS_File = SD.open("datalog.txt", FILE_WRITE);

      // If the file is available, write to it:
      if (GPS_File) {
        GPS_File.print(Buffer);
        GPS_File.close();
        // print to the serial port too:
        //Serial.print(Buffer);
      }  
      // If the file isn't open, pop up an error
      else {
        Serial.println("Error opening datalog.txt");
      }
      Buffer = "";
    } 
  }
  // Now do something else - but don't block here either
}

While there are GPS characters to read, it will process them but otherwise it does nothing. After the while loop you can do something else, e.g. blink a LED, but that code must not block either or it will interfere with reading the GPS message.

Pete
 
Pete,
Your tweak works, amazing! It now logs at 20Hz cleanly. Tested on the T4.1 just now. Thx for the suggestions!

$GNRMC,130049.75,A,2936.6095171,N,09506.5759320,W,0.050,,090520,,,A,V*06
$GNRMC,130049.80,A,2936.6095173,N,09506.5759320,W,0.027,,090520,,,A,V*0E
$GNRMC,130049.85,A,2936.6095172,N,09506.5759323,W,0.015,,090520,,,A,V*08
$GNRMC,130049.90,A,2936.6095163,N,09506.5759320,W,0.049,,090520,,,A,V*06
$GNRMC,130049.95,A,2936.6095165,N,09506.5759323,W,0.022,,090520,,,A,V*0B
$GNRMC,130050.00,A,2936.6095167,N,09506.5759320,W,0.018,,090520,,,A,V*07
$GNRMC,130050.10,A,2936.6095170,N,09506.5759326,W,0.034,,090520,,,A,V*08
$GNRMC,130050.20,A,2936.6095166,N,09506.5759332,W,0.021,,090520,,,A,V*0D
$GNRMC,130050.25,A,2936.6095165,N,09506.5759330,W,0.007,,090520,,,A,V*0D
$GNRMC,130050.35,A,2936.6095159,N,09506.5759335,W,0.029,,090520,,,A,V*0A
 
Status
Not open for further replies.
Back
Top