Fastest way to transfer data to/from Teensy 4.0/4.1 and Windows 10?

Status
Not open for further replies.

weedogt

New member
I need to transfer about 500 k of data from a PC to a Teensy 4.0/4.1 and then about the same amount back again to the PC.

I'm running Windows 10 in my PC and build apps with .NET.

What would be the fastest way to transfer data? It's OK to add som module if that will increase the speed. Would, for example, a FT232H board be useful?
 
Teensy's USB port is the best way. NativeEthernet on Teensy 4.1 would be an alternative if you really don't want to use USB.

FT232H probably isn't a great option, because then you would need very fast serial between the Teensy & FT232H. That might be possible, if your code is very efficient, or if you add extra buffers to the hardware serial class, or use RTS/CTS flow control. But even in a best case scenario, hardware serial has far more overhead than USB and Ethernet, which use efficient bus master DMA.
 
OK, I think I get it. Should I use the standard USB connector on Teensy, or should I add a second? And if so, should it be Host or Device?
 
OK, I think I get it. Should I use the standard USB connector on Teensy, or should I add a second? And if so, should it be Host or Device?

I've had good results transferring data at rates over 10MB/second using the USB Serial port. I've recently found that it makes things simpler to use the USB Dual Serial setup. I use SerialUSB1 for the data transfers and Serial to control the program. Both these USB links work over the same standard USB hardware connection. The tough part is writing the code on the PC to read the data from the SerialUSB1 port and buffer or store it. A fairly new PC can keep up with the SerialUSB1 data transfer without handshaking if you break up the transmission from the T4.X into blocks of 4 or 8KB. Apparently the transition between one block and the next adds just enough delay in the USB transfers to let the PC keep up. Of course your results may differ based on the speed of your PC and the code you are running. I wrote my PC host program in the free version of the Embarcadero C++ Builder system.
 
@mborgerson Excellent, thanks! What Teensy did you use?

10 MB/s should be enough for my little project. My PC is a few years old, it has a Intel Core i5-6300U CPU @ 2.40GHz with Win 10 Pro 64-bit. Is that in line with what what you have been using?

Perhaps you can give some pointers regarding your C++ code? Thanks again.
 
Rather than post hundreds of lines of C++ Builder code that you could build to illustrate data transfers that take only a few dozen lines, I bit the bullet and came up with a couple of Python scripts. The first script opens a file, then reads the data and sends it to the Teensy on one of the USB Serial connections. The T4.x stuffs the incoming data into a buffer (in DMAMEM in the example).

A second Python script asks the T4.x to return the data from the buffer and the script writes the data to a new file. I used a .bmp image file for testing because is was easy to check for errors by looking at the returned file. The Pythons scripts are very simple--I only have about 15 hours of Python programming experience--and a lot of that was spent stumbling around the internet looking for example code. The Teensy program that talks to the Python scripts is a bit more refined--but then I've got about a thousand times more hours with C and C++ than with Python.

This Python script sends the file to the Teensy 4.x:

Code:
# Send a file to Teensy in binary format
# I'm using a .bmp file in the example
# because it's easy to look at the returned data
# and see if it has survived the round trip to the
# Teensy and back

import serial
from os.path import getsize

## You will need to make sure which PC Comm Ports are
## used in your system and adjust the comm port numbers
## If you're using IOS or Linux,  I hope you know how to
## make the changes----I sure don't!
datport = serial.Serial('COM9', 115200, timeout=5)
ctlport = serial.Serial('COM10', 115200, timeout=3.5)
print("Serial ports open");

ctlport.write(b'r')  # tell the teensy that data is coming

fname = "test.bmp" # change the following for your own test file
binfile = open(fname, 'rb')
filesize = getsize(fname)
print("File Size: ", filesize)
datport.write(filesize.to_bytes(4,byteorder='little'))
## Read the whole file and write it in one line!   Nice!
datport.write(binfile.read())
binfile.close()

print("Sent " , fname);

##*****************************************************
##optional request to have teensy play back first part of
##buffer data.
##ctlport.write(b'p')  # tell the teensy to send buffer data
##print("Requesting buffer playback")

ch = b' '
while(ch.decode() != 'q'):
    ch = ctlport.read(1)
    print(ch.decode(), end='')
    
print("Program ended with received <q>");
datport.close()
ctlport.close()
print("Serial ports closed.");


This Python script asks the Teensy 4.x to send back the data sent by the first script. It writes the received data to a different file for later comparison:
Code:
# Receive a data buffer from the Teensy and write to file
# I'm  receiving from a .bmp file in the example
# because it's easy to look at the returned data
# and see if it has survived the round trip to the
# Teensy and back

import serial
from os.path import getsize


## You will need to make sure which PC Comm Ports are
## used in your system and adjust the comm port numbers
## If you're using IOS or Linux,  I hope you know how to
## make the changes----I sure don't!
datport = serial.Serial('COM9', 115200, timeout=0.5)
ctlport = serial.Serial('COM10', 115200, timeout=3.5)
print("Serial ports open");

ctlport.write(b's')  # tell the teensy to send data

fname = "rcvtest.bmp"  # get back 154KB QVGA RGB565 test file
binfile = open(fname, 'wb')
blocksize = 4096
## Teensy will tell us how many bytes to expect
mybytes = bytearray([0,0,0,0])
## Read 4 bytes from Teensy to get # bytes to expect
mybytes = datport.read(4)
filesize = int.from_bytes(mybytes,"little")

bytesleft = filesize

print("Data length is ",filesize, " bytes")
      
while (bytesleft > 0):
    if bytesleft > blocksize:
        bytestoread = blocksize
    else:
        bytestoread = bytesleft
    ## Read and write a block all in one line!   Nice!
    binfile.write(datport.read(bytestoread))
    bytesleft = bytesleft - bytestoread
        
binfile.close()

## Now we read and display anything the Teensy sent before <q>
ch = b' '
while(ch.decode() != 'q'):
    ch = ctlport.read(1)
    print(ch.decode(), end='') # no EOL after each character
print("Program ended with received <q>");
datport.close()
ctlport.close()
print("Serial ports closed.");

print("Saved to " , fname);

datport.close()
ctlport.close()
print("Serial ports closed.");

Here is the Teensy sketch that responds to the PC and transfers data over the second USB serial connection:

Code:
/**************************************
 * Minimal binary transfer using 2 serial channels
 * control channel receives requests to send or receive
 * data channel sends or receives binary data
 * This program is written for T4X and saves buffered
 * data in either DMAMEM or EXTMEM
 */
const int ledpin  = 13;
#define  LEDON   digitalWriteFast(ledpin, HIGH);
#define  LEDOFF  digitalWriteFast(ledpin,  LOW);

#define BUFFSIZE 300*1024

uint8_t buffer[BUFFSIZE] DMAMEM;
 
elapsedMillis trtimer = 0;

void setup() {
  Serial.begin(9600); // the control channel
  SerialUSB1.begin(9600);  // the data channel
  delay(400);
  pinMode(ledpin, OUTPUT);
  Serial.println("Sending q"); 
  memset(buffer, 0x44,BUFFSIZE);
}
// This global gets set by RcvBuffer call
uint32_t rcvbytes = 100;
void loop() {
  char ch;
   if (Serial.available()) {
    ch = Serial.read();
    switch (ch) {
      case 's' :
        SendBuffer(rcvbytes);
        break;
      case 'r' :
        rcvbytes = RcvBuffer();
        break;
    } // end of switch(ch)

  }  // end of if(Serial.available())
}

void SendBuffer(uint32_t numbytes){

  uint32_t datlen, endmillis;
  float frate;
  LEDON
  trtimer = 0;
  datlen = numbytes;
  // Tell the PC how many bytes to expect
  SerialUSB1.write((uint8_t *)&datlen,4);
  // Now send the bytes from the buffer
  // I write the whole thing in one chunk---just because I can!
  SerialUSB1.write((unsigned char *)&buffer[0], datlen);
  LEDOFF
  endmillis = trtimer;
  Serial.printf("Teensy Sent %lu bytes in %lu mSec ", datlen,endmillis);
  frate = (float)datlen/1024.0/(endmillis /1024.0);
  Serial.printf(" or %6.1f KBytes/Second\n", frate);
  Serial.println("q"); 
}


// RcvBuffer expects a 4-byte unsigned long indicating the 
// number of bytes to receive, then that many following
// bytes.  The data is stored in buffer.
#define TIMEOUT 2000
uint32_t RcvBuffer(void){
  uint32_t datlen, dcount, endmillis;
  float frate;
  dcount = 0;
  do {
    delay(1);
    dcount ++;
  } while((dcount <TIMEOUT) && (SerialUSB1.available()<4));
  if(dcount >= TIMEOUT) {
    Serial.println("Timeout waiting for data\n");
    Serial.println("q"); // tells PC that we're done
    return 0;
  }
  LEDON
  trtimer = 0;
  SerialUSB1.readBytes((char *)&datlen,4);
  if(datlen > BUFFSIZE) datlen = BUFFSIZE;
  // now read the data into buffer
  SerialUSB1.readBytes((char *)&buffer, datlen);
  endmillis = trtimer;
  LEDOFF
  Serial.printf("Teensy Received %lu bytes in %lu mSec. ", datlen,endmillis);
  frate = (float)datlen/1024.0/(endmillis /1024.0);
  Serial.printf(" or %6.1f KBytes/Second\n", frate);
  
  // Uncomment the next line to have the teensy echo back 200 bytes
  // PlayBuffer();
  Serial.print("\nq");// Tell PC we're done
  return datlen;
}


// Send back some buffer contentsin hex format over the control USB Serial link
void PlayBuffer(void){
  uint16_t i;
  for(i= 0; i< 200; i++){
    Serial.printf("%02X ",buffer[i]);
  }
  Serial.println("Playback done. q");
}

The Teensy sketch sends back some timing data on the transfers---but I'm not sure how how much to rely on the indicated speeds. Both the Teensy 4.x and the PC could be moving data around with DMA many milliseconds after they tell you they are finished.

Here's what the Idle shell reported with one set of transfers:
Code:
===================== RESTART: C:\Users\User\Python39\sendfile.py =====================
Serial ports open
File Size:  154178
Sent  test.bmp
Teensy Received 154178 bytes in 5 mSec.  or 30835.6 KBytes/Second

qProgram ended with received <q>
Serial ports closed.
>>> 
====================== RESTART: C:/Users/User/Python39/rcvfile.py =====================
Serial ports open
Data length is  154178  bytes
Teensy Sent 154178 bytes in 4 mSec  or 38544.5 KBytes/Second
qProgram ended with received <q>
Serial ports closed.
Saved to  rcvtest.bmp
Serial ports closed.
>>>

Now that I've satisfied my curiosity about whether Python can be used for USB data transfers, I'll see if I can write similar routines to do the same thing in C++builder without all the GUI overhead. It's been a decade or more since I wrote a non-GUI app with C++builder, so that ought to be an interesting experience.
 
I'm running Windows 10 in my PC and build apps with .NET.

@weedog: Here here some dotNet examples: https://github.com/TeensyUser/doc/wiki/Serial I tested communication with dotNet up to some 15MByte / sec (T4.0). There also is a corresponding forum thread describing this test.

@mborgerson: Mind if I copy your python script to the wiki page (link above)? It currently only has C# examples for serial communication to the PC...
 
Status
Not open for further replies.
Back
Top