Teensy 4.1 strange serial port behavior (USB serial vs. serial1, serialN)

nickrehm

Member
I am writing serial data from a teensy 4.1 and reading it from a windows machine using python and pyserial.

On the teensy side, I have initialized the serial port and am writing a data packet with a header and checksum:

Code:
  unsigned char checksum = XORChecksum((uint8_t *) &telemPacket, telemPacketLength); // Create checksum
  Serial.write(msg_header); // send packet header
  Serial.write((uint8_t *) &telemPacket, telemPacketLength); // Packet
  Serial.write((uint8_t *) &checksum, 1); // Send checksum

All together I am sending 82 bytes of data in a 100hz loop. Serial baud rate is set to 2000000. On the "receive" side using pyserial, I am simply checking the length of the input buffer in a 100hz loop and clearing it:

Code:
    print(ser.in_waiting)
    print(ser.read_all())
    ser.reset_input_buffer()


I am experiencing two distinct serial buffer behaviors, depending on if I am using the usb Serial, i.e. Serial.write() vs. the teensy hardware serial, i.e. Serial1.write(), Serial2.write(), etc.

USB serial setup and behavior:
  • microUSB from teensy to my windows machine running the pyserial script
  • pyserial output looks like this:

print(ser.in_waiting) --> 82 (sometimes 164 if my send/receive de-sync)
print(ser.read_all()) --> my packet
ser.reset_input_buffer()

  • this is behavior I would expect!



Teensy hardware serial setup and behavior:
  • USB/TTL cable hooked up to teensy 4.1 TX1, RX1 from teensy to my windows machine running the pyserial script
  • pyserial output looks like this:

print(ser.in_waiting) --> 0
print(ser.read_all()) --> times out and doesn't read anything because nothing in buffer
ser.reset_input_buffer()
.
.
.
This repeats ~48 times
.
.
.
print(ser.in_waiting) --> 3968
print(ser.read_all()) --> a bunch of my packets
ser.reset_input_buffer()
.
.
.


It appears that the teensy hardware serial is buffering until the output buffer is full, then that is being sent all at once. This doesn't happen with the usb serial. The result on the receive side is not seeing any data in the input buffer for many loop iterations, then getting ~48 packets all at once. Using Serial1.flush() does not change this behavior. I am not using flow control.
Am I going crazy and horribly misunderstanding serial communication basics? Or is this a known behavior? How would I emulate the behavior of the usb serial buffer?
 
Sorry I am not sure exactly what you are asking, So will try to give some generic information.

The Serial object, tries to fill up a USB buffer and then sends a USB packet to the host. Things that control how USB Serial buffers it.
On T4.1 running in High speed mode (default), The buffers are 512 bytes long. So if you send more than 512 bytes it will send out the full packet, to the host.
Or if a a certain amount of time elapses without seeing any more data, it will send what it has. Likewise, the Serial.flush() will send what it has at the time. When the host receives the USB packet, things like PySerial will probably see all of the characters of the packet at once.

Side note: The Baud rate like: Serial.begin(115200); does not change anything as far as the USB communications. It can have some meaning, for example if your sketch is a USB to Serial adapter and the host changes the logical baud rate. Your sketch can detect that and change the Hardware Serial port you are using as part of the sketch. See the USB Serial example for how this works.

Hardware Serial ports, is different. The Baud rate of 2,000,000 actually times how the characters are Serialized out. Each bit goes out one after another. Each character outputs like 10 bits of data. SO 2mbs can output 200k bytes per second. So yes the host will see a gap between characters of something like 1/200K seconds. (5us if quick calculation).
So your host script could see several iterations before it sees the next character.

However it also may depend on how are you hooking up the RX1/TX1 to the host. Does it go directly to the host? i.e. the host has a TTL Serial hookup? Or are you using a USB to Serial adapter? If so that adds another variable to the equation, as then it depends on how that USB to Serial adapter works. I have seen some that send one byte per packet, and others that buffer up the data and reduce the number of USB packets.

Not sure if this helps or not.
 
Thanks for the input! What you describe with the hardware serial ports is exactly what I'm not observing; I would expect that writing to the hardware serial port once every 1/100 of a second would lead to a 100hz read loop seeing new data in the receive buffer basically every iteration (assuming the two loops are ~close in speed). What I actually observe is only seeing data on the receive side every ~1/2 second, and when there is data, its 3968 bytes worth. i.e. many many packets that should have instead arrived one at a time in the 100hz loop come in all at once.

This is the USB cable I am using to hook up to hardware serial: https://www.amazon.com/Serial-Adapt...lpcontext&ref_=fplfs&smid=A2VPXM5R5067ZO&th=1 I don't think there is anything under the hood there. I am only hooking up tx, rx and ground
 
Strong guess, FTDI cable is buffing the stuff...
If you output your 82 bytes to userial1 and then do userial1.flush();
Does it work? My guess maybe not.

The cable probably has some logicai IOCTL that tells the device to flush it's internal buffers.

If you want to verify the issue is there you might try a different cable that is not FTDI or use a teensy running the sketch I mentioned.
 
You could try adjusting the Advanced port settings:
2023-03-25 10_09_02-serial-advanced.png
This is an FTDI cable on my Windows 10 PC, I can't recall if these are the default settings or I've adjusted them myself in the past.
 
You could try adjusting the Advanced port settings:
View attachment 30713
This is an FTDI cable on my Windows 10 PC, I can't recall if these are the default settings or I've adjusted them myself in the past.

I think you've hit the nail on the head. The advanced settings on the USB serial port on the Windows dialog seem to indicate that it will try to buffer 4096 characters or wait until there is a gap of 16 milliseconds in the incoming data. That 16mSec gap will never occur with one data packet every 10mSec. (If I'm understanding what the 'latency timer" ) Correctly. The FTDI Knowledge Base has this to say:
The latency timer is a form of time-out mechanism for the read buffer of FTDI devices. When a FT_Read instruction is sent to the device, data will not be sent back to the host PC until the requested number of bytes has been read. If the requested number of bytes never comes, the device would not send data back.

The latency timer counts from the last time data was sent back to the PC. If the latency timer expires, the device will send what data it has available to the PC regardless of how many bytes it is waiting on. The latency timer will then reset and begin counting again.

The default value for the latency timer is 16ms. This value may be customised by adding or changing the following entries in the FTDIPORT.INF file of the driver before installation.

[FtdiPort232.NT.HW.AddReg]
HKR,,"LatencyTimer",0x00010001,50

This example will set the default latency timer value to 50ms. The valid range for the latency timer is 1ms - 255ms, although 1ms is not recommended as this is the same as the USB frame length.

The latency timer value is held in the registry under

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\FTDIBUS\{Device VID, PID and serial number}\0000\Device Parameters\LatencyTimer

Please see AN232B-04 Data Throughput, Latency and Handshaking for additional information on the latency timer.


Setting the latency time to 5mSec should have the Teensy serial output forwarded to the USB port on the PC after each packet since it takes less than 1/2 mSec to send an 82-byte packet and the serial input will be quiet for 9.5mSec.

Even if this is the issue, You can't depend on the PC to make the USB data available to PySerial immediately, so you can't count on PySerial getting the data at exactly 10mSec intervals.
 
The AN232B-04 app note is a pretty good explanation of what is going on inside the FT232. Just search for "FTDI AN232B-04". It should be mandatory reading for anyone whose expectations of the serial-to-USB converters don't match the results they experience.
 
Thanks all, this is very good information and I think is exactly the "problem" I am seeing. Well, less of a problem and more of my lack of fully understanding. I will troubleshoot on Monday when I can get my hands on the hardware again and report back.
 
Back
Top