Sending 180k bytes over Serial

[2over12]

Hi,

I've been messing around with an application of the Teensy 4.0 where it receives some bytes and replays them back with some modifications. I've been running into issues where if the writer writes a lot of bytes quickly the program hangs. I've reduced everything to an MWE where 180,000 to the serial port and waiting to read the 180,000 bytes.

use std::{
    fs::{File, OpenOptions},
    io::{Read, Write},
};
const BUFFER_SIZE: usize = 180_000;
use rand::prelude::*;
fn main() {
    let mut f = OpenOptions::new()
        .read(true)
        .write(true)
        .open("/dev/ttyACM0")
        .unwrap();
    let mut send_buffer: [u8; BUFFER_SIZE] = [0; BUFFER_SIZE];
    let mut recv_buffer: [u8; BUFFER_SIZE] = [0; BUFFER_SIZE];
    let mut rng = thread_rng();
    rng.fill_bytes(&mut send_buffer);
    f.write_all(&send_buffer).unwrap();
    f.read_exact(&mut recv_buffer).unwrap();
    assert!(send_buffer == recv_buffer);
    println!("Hello, world!");
}

Then on the teensy side I simply have:

// As simple as it gets. Just echo back what's seen over USB serial.
void setup() {
  Serial.begin(115200); // baud doesn't matter (native USB).
}
void loop() {
  while(Serial.available()){
    Serial.write(Serial.read());
  }
}

On my linux box this will hang forever. The write finishes but the read never completes, at less than 180k say 150k everything goes through. It was my understanding that USB flow control should take care of this type of thing and I would not need any special rate limiting on the host side.

It's not super clear to me what is occurring here because the write terminates but the read never completes. My first though is that there is some limit on how much data can be buffered for a read on the linux side. My expectation would be that once this buffer was full, the writes on the teensy side would block causing the writes on the linux side to block. Instead, only the read blocks.

Ideally on the user side we just want to expose a library function that looks basically like a file descriptor where you can dump an arbitrary amount of bytes into it and later call read on it when the user is ready. I thought that should be possible with USB but perhaps not. Is the only solution here to basically fragment it where the client is basically only allowed to have say 10k bytes in flight at a time and then has to wait for an acknowledgment that those 10k bytes have been processed before sending the next fragment.

I guess the other option would be potentially have some sort of service on the pc responsible for constantly reading off the serial port and managing its own buffer (assuming that a buffer on the pc is causing this process to block). This solution would allow us to deal with growing the buffer instead of causing the transfer to seize up.

Thanks for any advice on how to get this done!

 

[defragster]

Make this change and say what happens:

void setup() {
  Serial.begin(115200); // baud doesn't matter (native USB).
  while( !Serial );  // wait until Serial is online
  // while( !Serial && millis() < 4000 );  // just wait 4 seconds

}

 

[2over12]

Thanks for the reply! That change results in the same behavior described above.

Doing some funky thread stuff results in a 10MB transfer going through.

use std::thread;
use std::{
    fs::{File, OpenOptions},
    io::{Read, Write},
};
const BUFFER_SIZE: usize = 10_000_000;
use rand::prelude::*;
fn main() {
    let mut fw = OpenOptions::new().write(true).open("/dev/ttyACM0").unwrap();
    let mut fr = OpenOptions::new().read(true).open("/dev/ttyACM0").unwrap();
    let mut send_buffer: Vec<u8> = vec![0; BUFFER_SIZE];
    let thrd = thread::spawn(move || {
        let mut recv_buffer: Vec<u8> = vec![0; BUFFER_SIZE];
        fr.read_exact(&mut recv_buffer).unwrap();
        recv_buffer
    });
    let mut rng = thread_rng();
    rng.fill_bytes(&mut send_buffer);
    fw.write_all(&send_buffer).unwrap();
    let recv_buffer = thrd.join().unwrap();

    assert!(send_buffer == recv_buffer);
    println!("Hello, world!");
}

So I think it does have something to do with a buffer on the linux side. Not totally sure.

 

[defragster]

Perhaps:

void setup() {
  Serial.begin(115200); // baud doesn't matter (native USB).
  while( !Serial );  // wait until Serial is online
  // while( !Serial && millis() < 4000 );  // just wait 4 seconds
}

void loop() {
  int ii=0;
  while(Serial.available() && ii < 512){
    Serial.write(Serial.read());
    ii++;
  }
  if ( ii ) Serial.flush();
}

 

[KurtE]

Sorry, not sure which language you are using on PC? C#? ... Which Linux Box? running?

Again sorry hard for me to give much of an idea on the PC side as not really sure of which language and things like, is the underlying code doing a lot of allocating memory and freeing... Is there a leak? Or is there hanging up with Garbage collection or ???

On Teensy side, I might do things in blocks, something like: Warning typed in on fly so probably typos...

void setup() {
  Serial.begin(115200); // baud doesn't matter (native USB).
  while( !Serial );  // wait until Serial is online
  // while( !Serial && millis() < 4000 );  // just wait 4 seconds
}

uint8_t buffer[512];
void loop() {
  int cb;
  while ((cb = Serial.available()) {
    if (cb > sizeof(buffer)) cb = sizeof(buffer);
    if (cb > Serial.availableForWrite()) cb = Serial.availableForWrite();
    Serial.read(buffer, cb);
    Serial.write(buffer,cb);
  }
}

Again not fully understanding your PC side, I believe it is sort of like in C++
for (; {
SendPacket();
ReceivePacket();
compare the two
}

So there should not be issue of running out of USB memory on Teensy, but maybe again some type of handshake issue, or count of characters sent/received not same or ???

 

[mlu]

To know where the problem occurs, count the number of received characters on the Teensy side, connect a serial adapter or another Teensy to the HW serialport to receive debug/status information.

 

[2over12]

Yeah so sorry the linux side is a linux box that is trying to communicate to the teensy over USB. The client code is written in Rust (indeed the actual Teensy code that I'm working in is also in Rust but I made the MWE in Arduino to help communicate). The client code just writes BUFFER_SIZE random bytes to the teensy's tty (essentially eventually doing a basic write syscall until all bytes are written) and then attempts to read that number of bytes back. The trick is we need to support a user dumping a bunch of bytes to the teensy without reading them back ie. the 10 MB example above. The threaded example runs a reader in a parallel thread to the writer which allows the whole thing to complete, so I think the issue was that somewhere on the linux side a buffer was full, preventing the teensy from completing a write. By moving the read into a parallel thread the reader then reads bytes out of that buffer preventing the teensy from blocking (I think). That's my hypothesis on why the threaded version works anyways.

I am going to attempt to implement this solution with the full project where the actual Teensy code is written in Rust using https://github.com/mciantyre/teensy4-rs and RTIC and see if I can get something working.