Dropped data from T4.X but not T3.X using MATLAB to read output of serial.write()

Status
Not open for further replies.

J_Sanders

Active member
Hi,

I'm trying to nail down a USB data transmission issue I'm seeing with Teensy 4.0 and 4.1.
It has been a heck of a bug to chase.

Here's some background info:
-I'm using Windows 10 OS, Arduino 1.8.13 and Teensyduino 1.53 (I also tried 1.54 beta 5)
-The bug exists when receiving data from Teensy with some serial interface software but not others. Working: Python's PySerial. Not working: MATLAB's serial interface, and PsychToolbox IOPort.
-As noted previously, T3.6 does not have the issue - I'm only seeing this with with T4

Test design:
I wrote the following script to return 1MB of data via USB serial when Teensy receives a '?' byte:

Code:
#define BUF_SIZE 10000
#define N_BUF_WRITES 100

byte usbTransferBuffer[BUF_SIZE] = {0};
uint16_t val = 0;

void setup() {
  // Set up dummy data for diagnostics
  for (int i = 0; i < BUF_SIZE; i++) {
    usbTransferBuffer[i] = val;
    val++;
    if (val == 256) {
      val = 0;
    }
  }
}

void loop() {
  if (Serial.available() > 0) { // Wait for any byte
    byte inByte = Serial.read(); // Read the byte
    if (inByte == '?') {
      for (int i = 0; i < N_BUF_WRITES; i++) { 
          Serial.write(usbTransferBuffer, BUF_SIZE); // Write the buffer via USB serial
          //delay(10); // Uncommenting this line makes it work with the IOPort interface.
      }
    }
  }
}

On the PC side, I open a serial port, send the '?' byte to trigger transmission, attempt to receive 1MB, and compute its checksum.

Here's the PC side Python code that works:

Code:
import serial
ser = serial.Serial("COM3")
ser.write(b'?'); Msg = ser.read(1000000)
checksum = sum(Msg)

The interpreter returns almost instantly, and the checksum always reads a correct value of 127308000.

Here's the MATLAB side code that works with T3.6 and fails with T4:

Code:
s = serial('COM3', 'Timeout', 3, 'InputBufferSize', 1000000);
fopen(s);
fwrite(s, '?', 'uint8'); Msg= fread(s, 1000000, 'uint8');
checksum = sum(Msg)

On T4, not enough data is returned, specifically on the first try after T4 gets reset by software or a power cycle. Subsequent tries return enough data, but the incorrect checksum. Repeat tries eventually "settle" and the correct checksum is returned each time.

A few observations:
-Oddly, slowing down the transfer by uncommenting the 'delay(10)' line yields a correct checksum on the first run with the IOPort interface, and *sometimes* with MATLAB's native serial interface
-There is no loss when sending smaller amounts of data (e.g. 10KB)
-The data loss and/or corruption only occurs in the first few runs after Teensy 4 is reset. With repeat attempts, something on the PC side is fixed (a buffer resized somewhere?) and the checksum is correctly computed each time afterwards.

Things I've tried that don't work:
-Adding Serial.send_now() and/or Serial.flush() after each write()
-Writing only if Serial.rts() is true
-Configuring DTR and RTS to either state on the PC side
-Making smaller (+larger) buffers and doing more (+fewer) reads and writes on the PC and Teensy side
-Making a Teensy-side loop to send each byte with Serial.write(byteN) instead of writing from a buffer with Serial.write(buf, nBytes)

A few thoughts:
One possibility is that T4's USB data stream is too fast for some interfaces (i.e. MATLAB and IOPort but not Python) to handle without lost data, while T3's is slow enough.
Another is that some interfaces don't play as nicely with the Windows serial driver, specifically as the driver is configured for T4 (but not T3).

I'm going to try to set this test up for some other interfaces (Julia, C++) to find out whether other interfaces are affected.

Any helpful suggestions are more than welcome!
 
> T4's USB data stream is too fast for some interfaces

sounds plausible.

Not only plausible, but likely. The T4.x USB serial interface is capable of transmitting about 18-20MBytes/second, while the T3.X interfaces are limited to about 1MByte/second. I was able to transfer large files to my PC by breaking the file up into chunks of 4KBytes and having the Teensy wait for an ACK character from the PC after each chunk is sent.
 
Status
Not open for further replies.
Back
Top