Serial2 Murder - UART Communications between boards

Status
Not open for further replies.

Davidelvig

Well-known member
Now that I have your attention...

I'm new to inter-board UART communication.
I'm testing 2 Teensy 3.2's using Serial2 on each, and the attached code.

Each works flawlessly, delivering 11.5Kbytes/seconds bi-directionally when hooked up as a loop-back (Rx2-Tx2 on the same board).

When connected to each other (cross-connected), it works sometimes flawlessly (similar speed), and other times one side detects erroneous bytes from the other.
(line 50 of the attached code detects a continuous stream of bytes >= 0x80, and all sent bytes should be < 0x80.)

I'm using HardwareSerial (Serial2) in both cases, using only Rx (9) and Tx (10) pins, plus a common ground (GND).
Both Teensy's are connected to their own MacBook via USB and share the very same codebase (shared on OneDrive)
I have not implemented RTC and CTS lines.

I must be missing a UART hand shaking phase. I think the second Teensy to start seems to be the one that fails in receipt of good bytes.
Perhaps byte receipt for the 2nd-to-start Teensy starts half-way through an inbound byte?

1) Is this an obvious and predictable problem with this setup?
2) Is there a good resource for learning about inter-board comms using UART?

Thanks
- Dave

p.s. My end goal is to communicate between my Teensy and either an nRF52 or ESP32 board for BLE comms. After I figure this out between 2 Teensy's

Code:
#include <Arduino.h>
#include <TeensyID.h>

bool useCapitals = false;

void getDeviceID_Byte(void) {
    // The following has a good chance of getting a device-unique ID byte - using a long MCU chip ID on Teensy
    uint8_t deviceID_Byte;
    uint32_t uid[4];
    kinetisUID(uid);
    int32_t seedValue = uid[0] | uid[1] | uid[2] | uid[3];
    randomSeed(seedValue);
    deviceID_Byte = random(0, 127);
    Serial.printf("deviceID_Byte = %d\n", deviceID_Byte);
    if (deviceID_Byte == 61) useCapitals = true; // determine which of 2 participating Teensy's should use capital letters.  Your number will vary
}

void setup() {
    Serial.begin(9600);     
    while (!Serial && millis() < 1000) {;}  
    delay(500); 
    Serial.printf("Serial start-up at %dms\n", millis());

    Serial2.begin(115200);                                          
    delay(500); Serial.printf("Serial2 start-up at %dms\n", millis());
    while (Serial2.available() > 0) Serial2.read();     //empty the inbound buffer

    getDeviceID_Byte();
}

unsigned long delayCount = 0;
unsigned long lastReportMS = 0;
unsigned long goodRecBytes = 0;
unsigned long badRecBytes = 0;
unsigned long lastRecBytes = 0;
unsigned long sendBytes = 0;
unsigned long lastSendBytes = 0;

byte baseByte = 0;
byte lastByte = 0;
void loop() {
    unsigned long millisNow = millis();
    int inboundAvail = Serial2.available();
    int outboundAvail = Serial2.availableForWrite();
    // receive
    if (inboundAvail > 0) {
        byte inByte = Serial2.read();
        if (inByte >= 0x80)  {
            inboundAvail = Serial2.available();
            while ((inByte >= 0x80) && (inboundAvail > 1)) {
                Serial.printf("%lu\tinByte too high: %d\tbytes available: %d\n", micros(), inByte, inboundAvail);            
                inByte = Serial2.read();
                inboundAvail = Serial2.available();
            }
            Serial.println("_________________________");
        }
        if ((inByte != lastByte + 1) && (inByte != (lastByte - 25)) && (lastByte != 0)) {
            badRecBytes++;
            Serial.printf("%d\t%d\n", lastByte, inByte);
        } else {
            goodRecBytes++;
        }
        lastByte = inByte;
    } 

    // send
    if (outboundAvail > 10) {
        byte outByte = (useCapitals) ? 'A' : 'a';
        outByte += baseByte;
        Serial2.write(outByte);
        sendBytes++;
        baseByte++; if (baseByte > 25) baseByte = 0;
    } else {
        Serial.printf("outboundAvail: %d\n", outboundAvail);
        // delay(10);
        delayCount++;
    }

    // print stats
    if (millisNow > (lastReportMS + 1000)) {
        Serial.print(useCapitals ? "CAPS\t" : "small\t");
        Serial.printf("%10dms\tgoodIn: %lu\tbadIn: %lu\tsent: %lu\tdelayK: %lu\tin rate: %lu bytes/sec\tout rate: %lu bytes/sec\n", 
                       millisNow, goodRecBytes, badRecBytes, sendBytes, delayCount/1000, 
                       ((badRecBytes + goodRecBytes) - lastRecBytes), (sendBytes - lastSendBytes));
        lastReportMS = millisNow;
        lastSendBytes = sendBytes;
        lastRecBytes = badRecBytes + goodRecBytes;
    }
}
 
I only read your code quickly, but my 1st impression is you may be transmitting 100% bandwidth on the serial line. Looks like you're always write more data when the buffer can take 10 chars and nothing else in your code is delaying, which should result in 100% bandwidth usage on the serial line.

If you do that, and if you start the receiver "in the middle" of 100% bandwidth serial, you can run into this somewhat obscure sync issue.

https://forum.pjrc.com/threads/6745...r-1200-baud-rate-for-UART?p=281054#post281054
 
Sounds like the issue. Thanks!

I am indeed at full speed here, and it won't be "in real life".. That is, I can go slower.

You mention in the other post a "10 microSecond delay every 5000 messages"
I was not able to get it to work with that light a touch.
My new code is here. It works if I add a delayMicroseconds(100) after each byte sent, but not with delayMicrosecond(75).
This results in 9100 bytes per seconds, which is sufficient.

I also tried this at 57600 bd.
It failed with a 100 uS delay, and succeeded with a 200uS delay - resulting in 4765 bytes/second.

I would like to find a "definitive" way to assure synchronization, and I have bandwidth to play with.

1) Maybe a specific slow-speed handshake (long inter-byte delay) until both are connected, then speed up?
2) maybe I detect the error on the receiving side (as I do here) and send a "re-synch request" message back to the sender (which I don't do yet)?
3) Maybe a slower baud rate (does it make sense that the reduction in baud rate above required a longer inter byte delay?)

I'd appreciate any additional guidance.

- Dave

Code:
#include <Arduino.h>
#include <TeensyID.h>

bool useCapitals = false;

void getDeviceID_Byte(void) {
    // The following has a good chance of getting a device-unique ID byte - using a long MCU chip ID on Teensy
    uint8_t deviceID_Byte;
    uint32_t uid[4];
    kinetisUID(uid);
    int32_t seedValue = uid[0] | uid[1] | uid[2] | uid[3];
    randomSeed(seedValue);
    deviceID_Byte = random(0, 127);
    Serial.printf("deviceID_Byte = %d\n", deviceID_Byte);
    if (deviceID_Byte == 61) useCapitals = true; // determine which of 2 participating Teensy's should use capital letters.  Your number will vary
}

void setup() {
    Serial.begin(9600);     
    while (!Serial && millis() < 1000) {;}  
    delay(500); 
    Serial.printf("Serial start-up at %dms\n", millis());

    Serial2.begin(115200);                                          
    delay(500); Serial.printf("Serial2 start-up at %dms\n", millis());
    while (Serial2.available() > 0) Serial2.read();     //empty the inbound buffer

    getDeviceID_Byte();
}

unsigned long delayCount = 0;
unsigned long lastReportMS = 0;
unsigned long goodRecBytes = 0;
unsigned long badRecBytes = 0;
unsigned long lastRecBytes = 0;
unsigned long sendBytes = 0;
unsigned long lastSendBytes = 0;

byte baseByte = 0;
byte lastByte = 0;
#define INTER_BYTE_DELAY 100
void loop() {
    unsigned long millisNow = millis();
    int inboundAvail = Serial2.available();
    int outboundAvail = Serial2.availableForWrite();
    // receive
    if (inboundAvail > 0) {
        byte inByte = Serial2.read();
        if (inByte >= 0x80)  {
            inboundAvail = Serial2.available();
            while ((inByte >= 0x80) && (inboundAvail > 1)) {
                Serial.printf("%lu\tinByte too high: %d\tbytes available: %d\n", micros(), inByte, inboundAvail);            
                inByte = Serial2.read();
                inboundAvail = Serial2.available();
            }
            Serial.println("_________________________");
        }
        if ((inByte != lastByte + 1) && (inByte != (lastByte - 25)) && (lastByte != 0)) {
            badRecBytes++;
            Serial.printf("%d\t%d\n", lastByte, inByte);
        } else {
            goodRecBytes++;
        }
        lastByte = inByte;
    } 

    // send
    if (outboundAvail > 10) {
        byte outByte = (useCapitals) ? 'A' : 'a';
        outByte += baseByte;
        Serial2.write(outByte);
        sendBytes++;
        delayMicroseconds(INTER_BYTE_DELAY);
        baseByte++; if (baseByte > 25) baseByte = 0;
    } else {
        Serial.printf("outboundAvail: %d\n", outboundAvail);
        delayCount++;
    }

    // print stats
    if (millisNow > (lastReportMS + 1000)) {
        Serial.print(useCapitals ? "CAPS\t" : "small\t");
        Serial.printf("%10dms\tgoodIn: %lu\tbadIn: %lu\tsent: %lu\tdelayK: %lu\tin rate: %lu bytes/sec\tout rate: %lu bytes/sec\n", 
                       millisNow, goodRecBytes, badRecBytes, sendBytes, delayCount/1000, 
                       ((badRecBytes + goodRecBytes) - lastRecBytes), (sendBytes - lastSendBytes));
        lastReportMS = millisNow;
        lastSendBytes = sendBytes;
        lastRecBytes = badRecBytes + goodRecBytes;
    }
}
 
You mention in the other post a "10 microSecond delay every 5000 messages"
I was not able to get it to work with that light a touch.

In that code the baud rate was 1 Mbit/sec. The goal is line idle for at least 9 bit times, though 10 is best.

For 115200, each bit is 87us. Use an idle of at least 870us to get 10 bit times.
 
Got it. I'll try a whole millisecond delay, and stretch out the number of bytes between such delays.
 
OK,
Delay of 1ms (1000uS) works with a delay every 12 bytes, but not every 15 bytes.
Good throughput at nearby 11 Kbytes/second.
Seems like a reasonable fix.
After the two boards get "in synch", it looks like they don't fail thereafter.
A million bytes so far in each direction with no errors. I'll monitor it.

And now to see how the ESP32 behaves as a partner

Thanks, @PaulStoffregen!
 
Interesting,
Attaching an ESP32 as the second device - same code, requires:
- dropping the BYTES_PER_DELAY from 12 to 2 (so a delay every 2 sent bytes); and
- increasing the INTER_BYTE_DELAY to 2000 uS (every second byte).
At that, I get about 990 bytes per second. Not really fast enough. And even then, about 1 in 5000 erroneous bytes.
All the errors show on the ESP side, indicating an unexpected byte value received from the Teensy.

Maybe I need to try SPI - a learning curve to get the ESP32 to act like a SPI Slave.

I'll post another thread asking for best practices for adding BLE MIDI (and BLE Serial) to a Teensy project.
 
Delay of 1ms (1000uS) works with a delay every 12 bytes, but not every 15 bytes.

Something else is going wrong. Once they get into sync, they should remain in sync. As you can see on that other thread, a 10 bit time delay was added only once every 5000 multi-byte messages.
 
Well that's annoying - and perhaps encouraging. Two emotions at once!
I'll keep looking - perhaps after I see if SPI Slave on ESP32 is easy.

I can try "getting in synch" the first time on the two-Teensy setup, and see if I can back down the pauses to a much longer interval.
 
Update:
This simplified code still works flawlessly on a loop-back-connected single Teensy, and fails on cross-connected dueling Teensy's

I can't figure out where the phantom / erroneous inbound bytes are coming from.
Could you have another look, Paul?
(Note: the "YOUR DEVICE WILL VARY" line)

Thanks!

Code:
#include <Arduino.h>
#include <TeensyID.h>

unsigned long receivedByteCount[256];
uint8_t deviceID_Byte;
bool sender;

// forward declarations
void startNonBlockingDelay(unsigned long duration);
bool isDelaying(void);
//

void setup() {
    Serial.begin(9600);     while (!Serial && millis() < 1000)  {;}    delay(500); 
    Serial.printf("Serial start-up at %dms\n", millis());

    Serial2.begin(115200);  while (Serial2.read() > -1)         {;}    delay(500);  //empty the inbound buffer
    Serial.printf("Serial2 start-up at %dms\n", millis());

    uint32_t uid[4];
    kinetisUID(uid);
    deviceID_Byte = uid[2];
    Serial.printf("deviceID = %d\n", deviceID_Byte);
    sender = (deviceID_Byte == 16);                              // YOUR DEVICE WILL VARY - run once to see value of deviceID_Byte


    memset(receivedByteCount, 0, sizeof(receivedByteCount));
}

unsigned long lastMillis = 0,
              sentBytes = 0,
              goodRecBytes = 0,
              badRecBytes = 0;
byte          b = 'B'; // decimal 66
void loop() {
    if (isDelaying()) return;

    int avail = Serial2.available();
    while (avail > 0) {
        int inByte = Serial2.read();
        if (inByte == b) goodRecBytes++;
        else             badRecBytes++;
        receivedByteCount[inByte]++;
        avail = Serial2.available();
    }
    if (sender) {
        if (Serial2.availableForWrite() > 10) {  // arbitrary capacity
            Serial2.write(b);
            sentBytes++;
        }
    } 
    unsigned long millisNow = millis();
    if (millisNow > (lastMillis + 1000)) {
        if (sender) startNonBlockingDelay(1000); // delay at least 9 bit-intervals to allow resynch of connected serial ports if needed.
        Serial.printf("\t\tSent: %lu\tGood In: %lu\tBad In: %lu\n", sentBytes, goodRecBytes, badRecBytes);
        Serial.println("Received bytes:");
        for (int x = 0; x < 256; x++) {
            if (receivedByteCount[x] > 0)
                Serial.printf("%3d: %lu\n", x, receivedByteCount[x]);
        }
        lastMillis = millisNow;
    }
}

// ----------- Utility functions ---------------

bool _delaying = false;
unsigned long delayStartMicros = 0;
unsigned long _duration;
void startNonBlockingDelay(unsigned long duration) {
    if (_delaying) {
        Serial.println("Error: Call to startNonBlockingDelay() while already delaying.");
    } else {
      _duration = duration;
      delayStartMicros = micros();
      _delaying = true;
    }
}

bool isDelaying(void) {
    if ((_delaying) &&
        (micros() > (delayStartMicros + _duration))) {
            _delaying = false;
    } 
    return(_delaying);
}
 
Picture of setup - each Teensy USB-connected to its own MacBook running Teensyduino 1.5.3
dual teensy setup.jpg

(Coin cell and Audio Shield are non-contributing extras, I hope)
 
Representative Sender side serial monitor snippet:
Sent: 2213632 Good In: 0 Bad In: 52801
Received bytes:
222: 1
223: 23
239: 3
246: 1
251: 1
253: 501
254: 1927
255: 50344
Sent: 2225164 Good In: 0 Bad In: 53026
Received bytes:
222: 1
223: 23
239: 3
246: 1
251: 1
253: 505
254: 1933
255: 50559

Representative Receiver side serial monitor snippet
Sent: 0 Good In: 1818943 Bad In: 4257384
Received bytes:
9: 2776955
11: 4
13: 30
40: 1075188
42: 316
44: 65984
46: 10537
62: 1
66: 1818943
67: 21
73: 821
77: 1
82: 550
98: 111453
99: 82
108: 4
110: 1
114: 18813
115: 30
137: 168630
139: 1
141: 74
168: 3
172: 65
174: 10
201: 27767
205: 37
226: 2
233: 1
242: 3
Sent: 0 Good In: 1818943 Bad In: 4257384
Received bytes:
9: 2776955
11: 4
13: 30
40: 1075188
42: 316
44: 65984
46: 10537
62: 1
66: 1818943
67: 21
73: 821
77: 1
82: 550
98: 111453
99: 82
108: 4
110: 1
114: 18813
115: 30
137: 168630
139: 1
141: 74
168: 3
172: 65
174: 10
201: 27767
205: 37
226: 2
233: 1
242: 3
 
grounded.jpgThat was it!
Dang, I had it grounded in earlier iterations... when I must have had other code errors.

Error free now.

Thanks a bunch! My confidence and sanity are returning.
 
Interestingly, I need a bit longer pause to "resynch" the serial when sending to the ESP32. 1500uS works, though 1000uS did not reliably.

Funny how the world looks like a better place now!
 
Status
Not open for further replies.
Back
Top