Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 8 of 8

Thread: Stream Data from Serial1 to SD Card, Teensy 3.6

  1. #1
    Senior Member
    Join Date
    Oct 2017
    Location
    Houston
    Posts
    448

    Stream Data from Serial1 to SD Card, Teensy 3.6

    I do a lot of GPS testing, and I'd like to just capture NMEA data (RMC messages) as it streams to the SD card on Teensy 3.6 Serial1. I have great success using the Sparkfun OpenLog SD card with T3.2s and T4.0s, but now I'm trying to get it working on the built-in T3.6 SD card. My routine below works fairly well at 1Hz on the T3.6, putting one RMC message per line. But when I bump the output to 10Hz, everything gets written to the a single line because the last couple of characters of each RMC message seem to not get captured.

    Is there a library (or sketch) that just does what the OpenLog does, i.e. just simply captures the entire NMEA data cleanly as it streams to Serial?

    Thanks for any advice!


    Code:
    // The Code...
    
    #include <SPI.h>
    #include <SD.h>
    
    #define gpsPort Serial1
    
    File GPS_File;
    void setup() {
    
      Serial.begin(115200);
      gpsPort.begin(115200);
    
      delay(500);
      
      Serial.println("Am Here");
    
      if (!SD.begin(BUILTIN_SDCARD)) {
        Serial.println("Card failed, or not present");
        return;
      }
      Serial.println("Card initialized.");
    
      // Remove old file
      SD.remove("datalog.txt");
    
      // Create new file
      GPS_File = SD.open("datalog.txt", FILE_WRITE);
      GPS_File.close();
      
      //GPS_File.write("START\n");
    
      // Enable LED pin
      pinMode(ledPin, OUTPUT);
    
    }
    
    void loop() {
      if (gpsPort.available())
      {
        String Buffer = "";
        while (gpsPort.available())
        {
            char GPSRX = gpsPort.read();
            Buffer += GPSRX;
    
        }
         
        // Open the file
        GPS_File = SD.open("datalog.txt", FILE_WRITE);
    
        // If the file is available, write to it:
        if (GPS_File) {
          GPS_File.print(Buffer);
          GPS_File.close();
          // print to the serial port too:
          //Serial.print(Buffer);
        }  
        // If the file isn't open, pop up an error
        else {
          Serial.println("Error opening datalog.txt");
        } 
        
        
      }
    }
    Last edited by defragster; 05-08-2020 at 12:01 AM. Reason: add CODE blocking

  2. #2
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,156
    As a crude first step to let GPS data arrive perhaps:
    Code:
        while (gpsPort.available())
        {
            delay(1);
            char GPSRX = gpsPort.read();
            Buffer += GPSRX;
        }
    If the .available() catches the string first bytes - it will exhaust those received before the message completes. The 1ms will probably trigger once - maybe 2-3 times in this case depending on message length and will waste time - but this is just to test all else is well for simple logging. It will assure getting a complete message before logging to SD. "SD.h" writes can take 10 to 100 or more ms to complete. There is a new thread with decent logging code that buffer incoming data for more efficient and fewer writes. And uses an _isr() to log the data - this should be safe for .read() just transferring buffered data as well - noted below. But the code above will overflow Serial1 buffer on long SD writes. There are observed times in the linked thread below.

    If there is a terminator char on the incoming string keying on that would allow better response.

    User added code near the current thread end tested here to work well : Teensy-3-6-SD-Card-missing-time-in-milliseconds { posts #6 and #10 }
    There is an SD.h and SDFat.h implementation - SDFat is way faster to write. The dual buffers could read as you do byte by byte until one buffer is full - then mark that buffer to WRITE and switch to the other buffer as done in that code, and size the buffers in bocks of 512 bytes. That avoids file open close on each write as well. Then as in that code start a timer ( intervalTimer or the raw code there ) and pull out Serial1 data into the buffers. { current writes of 64KB SD buffers are taking 2.88 ms - up linearly from 0.77 ms when it was 16 KB } with rare spikes using SDFat and exFAT formatted flash card.

  3. #3
    Senior Member
    Join Date
    Nov 2012
    Posts
    1,526
    You aren't making sure that you've read the whole line before writing it.
    Try replacing this:
    Code:
        while (gpsPort.available())
        {
            char GPSRX = gpsPort.read();
            Buffer += GPSRX;
    
        }
    with this:
    Code:
        while(1) {
          while (gpsPort.available() < 1);
          char GPSRX = gpsPort.read();
          Buffer += GPSRX;
          if(GPSRX == '\n')break;
        }
    Pete

  4. #4
    Senior Member
    Join Date
    Oct 2017
    Location
    Houston
    Posts
    448
    Thx, I'll try both these and let you know what happens, I appreciate it.

  5. #5
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,156
    Pete's post is code for this note in prior post : If there is a terminator char on the incoming string keying on that would allow better response.

    But both will fail as noted over time with slow write speed of SD and Serial1 buffer getting over run when the write takes in excess of 100 ms at 10 Hz. Linked post and notes on T_4.1 Beta thread about speed observed today .

  6. #6
    Senior Member
    Join Date
    Oct 2017
    Location
    Houston
    Posts
    448
    Pete,
    Your trick works. Thx. Here's what I see now (on both the T3.6 and T4.1):

    1. 10Hz NMEA RMC messages write to SD card just fine. I can also tested doing other tasks in the loop (blinking LED) and still saved the RMC messages just fine.
    2. 20Hz NMEA RMC messages write to SD card just fine as well, if nothing else in the Void loop. If I add a small task, even just blinking LED, I get dropped characters at the end of the line.

    Low-cost GPS receivers are moving towards 40Hz rates, but right now I can get by with the 10Hz rate for what I'm doing. But I have to say that it seems that a simple RMC message should be able to be recorded at 20+ Hz with no issues. I'll have to go see if the Sparkfun OpenLog SD can do 20 Hz, as I've used it routinely for 10 Hz.

  7. #7
    Senior Member
    Join Date
    Nov 2012
    Posts
    1,526
    You'll have to rewrite the loop function if you want to do more than just read the RMC message. The way it is written now, if there are any characters available from the GPS, it hangs until it has written a whole line to the SD.
    Replace the entire loop() function with this:
    Code:
    String Buffer = "";
    void loop() {
      while (gpsPort.available() >= 1)
      {
        char GPSRX = gpsPort.read();
        Buffer += GPSRX;
        if(GPSRX == '\n') {
          // Open the file
          GPS_File = SD.open("datalog.txt", FILE_WRITE);
    
          // If the file is available, write to it:
          if (GPS_File) {
            GPS_File.print(Buffer);
            GPS_File.close();
            // print to the serial port too:
            //Serial.print(Buffer);
          }  
          // If the file isn't open, pop up an error
          else {
            Serial.println("Error opening datalog.txt");
          }
          Buffer = "";
        } 
      }
      // Now do something else - but don't block here either
    }
    While there are GPS characters to read, it will process them but otherwise it does nothing. After the while loop you can do something else, e.g. blink a LED, but that code must not block either or it will interfere with reading the GPS message.

    Pete

  8. #8
    Senior Member
    Join Date
    Oct 2017
    Location
    Houston
    Posts
    448
    Pete,
    Your tweak works, amazing! It now logs at 20Hz cleanly. Tested on the T4.1 just now. Thx for the suggestions!

    $GNRMC,130049.75,A,2936.6095171,N,09506.5759320,W, 0.050,,090520,,,A,V*06
    $GNRMC,130049.80,A,2936.6095173,N,09506.5759320,W, 0.027,,090520,,,A,V*0E
    $GNRMC,130049.85,A,2936.6095172,N,09506.5759323,W, 0.015,,090520,,,A,V*08
    $GNRMC,130049.90,A,2936.6095163,N,09506.5759320,W, 0.049,,090520,,,A,V*06
    $GNRMC,130049.95,A,2936.6095165,N,09506.5759323,W, 0.022,,090520,,,A,V*0B
    $GNRMC,130050.00,A,2936.6095167,N,09506.5759320,W, 0.018,,090520,,,A,V*07
    $GNRMC,130050.10,A,2936.6095170,N,09506.5759326,W, 0.034,,090520,,,A,V*08
    $GNRMC,130050.20,A,2936.6095166,N,09506.5759332,W, 0.021,,090520,,,A,V*0D
    $GNRMC,130050.25,A,2936.6095165,N,09506.5759330,W, 0.007,,090520,,,A,V*0D
    $GNRMC,130050.35,A,2936.6095159,N,09506.5759335,W, 0.029,,090520,,,A,V*0A

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •