I have a Teensy 4 and have it collect a bunch of data and then stream it back to the computer over usb via Serial.write(). For small data sets, this works fine, but if I send 100kB or more, I get some data loss where at least a few bytes get dropped. I've tried to do both send (Teensy) and receive (python on computer) in packets, but it hasn't made much of a difference. I thought that Serial.write() should be blocking if the USB buffer is full, but I also tried adding a check with Serial.availableForWrite(), but that didn't help either. If I slow down the sending with a delay in the Teensy send loop, everything works fine, so it seems like the Teensy isn't able to properly throttle the send rate based on what the host can keep up with. Am I wrong that I should be able to just dump data over Serial.write() and it will block as needed for the computer to get all the data? If not, is there a way to prevent data loss (other than a big delay in the sending)?
I have a pretty simple Teensy firmware and python script that shows the problem. Here's the python code to receive the data and check that it is right:
I've attached the full .ino sketch I'm using, but the sending part of the code is
Any suggestions or ideas on what is going on and/or work-arounds would be appreciated. I'm using Teensduino 1.55 and the receive side is python 3.8 (64 bit) on Windows 10.
Thanks!
I have a pretty simple Teensy firmware and python script that shows the problem. Here's the python code to receive the data and check that it is right:
Code:
import serial, struct
import numpy as np
comm = 6
wordsToGet = 1000000
ser = serial.Serial('COM{}'.format(comm),115200,timeout=2)
expectedData = np.array(range(wordsToGet))
expectedData = expectedData.astype(np.uint16)
ser.write('{}\n'.format(wordsToGet).encode())
data=b''
received = 0
while( wordsToGet != received):
packetSize = min(1024,wordsToGet-received)
packet = ser.read(packetSize*2) # 2 bytes per word
if (len(packet) != packetSize*2):
raise Exception('Error, got only {} bytes, but expecting {}'.format(len(packet),packetSize*2))
data += packet
received += packetSize
data = np.array(struct.unpack('H'*wordsToGet,data))
diffSquared = (data-expectedData)*(data-expectedData)
if (np.sum(diffSquared) != 0):
print('Error, data returned not as expected')
print(np.array(diffSquared))
else:
print('{} words successfully received'.format(wordsToGet))
del ser
I've attached the full .ino sketch I'm using, but the sending part of the code is
Code:
while ( localWordsToSend != localWordsSent) {
uint packetSize = min((uint) SENDSIZE,localWordsToSend - localWordsSent);
while ( (uint) Serial.availableForWrite() < (packetSize * sizeof(data[0])));
Serial.write( (uint8_t*) &data[localWordsSent], packetSize * sizeof(data[0]));
localWordsSent += packetSize;
//delayMicroseconds(2500); //necessary to avoid missed data
}
Any suggestions or ideas on what is going on and/or work-arounds would be appreciated. I'm using Teensduino 1.55 and the receive side is python 3.8 (64 bit) on Windows 10.
Thanks!