High bandwidth comm between Teensy4.1 and PC

B

bigdlin

Member
Dear all,

I am currently working on a biomedical recording device using an overclocked Teensy4.1 (960 MHz) on 64 channels at 7407 Hz. Each channel generates 2 bytes of data per sample. Including headers, the base 64 encoded data amount is 1,348,074 bytes per second. I use Serial.print to transmit data from Teensy4.1 to a C# UI. It could only handle a 10th of the data load. Arduino serial monitor seems to perform fine, but there is no way to record the raw data. PuTTY has the same problem as C# serial. How can I utilize the 480Mb/s USB speed on my PC? Can provide more information on request. Thanks!
 
bigdlin said:
Arduino serial monitor seems to perform fine, but there is no way to record the raw data
Click to expand...
You may want to try TyCommander. That tool is able to log the incoming serial data:

1705696961866.png


Paul
 
Thank you! It works very well.

Still, is there a way to make my C# inform program perform better? I have included the necessary source code bits below regarding creating and using the serial port.

C#: 
internal class SafeSerialPort : SerialPort
{
    private System.IO.Stream theBaseStream;

    internal new void Open()
    {
        try
        {
            base.Open();
            theBaseStream = BaseStream;
            GC.SuppressFinalize(BaseStream);
        }
        catch
        {

        }
    }

    internal new void Dispose()
    {
        Dispose(true);
    }

    protected override void Dispose(bool disposing)
    {
        if (disposing && (base.Container != null))
        {
            base.Container.Dispose();
        }
        try
        {
            if (theBaseStream.CanRead)
            {
                theBaseStream.Close();
                GC.ReRegisterForFinalize(theBaseStream);
            }
        }
        catch
        {
            // ignore exception - bug with USB - serial adapters.
        }
        base.Dispose(disposing);
    }
}

C#: 
internal static SafeSerialPort cereal;

C#: 
MEAHeadstageProgram.cereal.BaudRate = 1152000;
MEAHeadstageProgram.cereal.DataBits = 8;
MEAHeadstageProgram.cereal.Handshake = Handshake.XOnXOff;
MEAHeadstageProgram.cereal.Parity = Parity.None;
MEAHeadstageProgram.cereal.StopBits = StopBits.One;
MEAHeadstageProgram.cereal.DtrEnable = true;
MEAHeadstageProgram.cereal.RtsEnable = true;
MEAHeadstageProgram.cereal.ReadBufferSize = 2147483647;
MEAHeadstageProgram.cereal.WriteBufferSize = 2147483647;
MEAHeadstageProgram.cereal.WriteTimeout = 5000;
MEAHeadstageProgram.cereal.ReceivedBytesThreshold = 1;
MEAHeadstageProgram.cereal.ReadTimeout = 5000;

MEAHeadstageProgram.cereal.PortName = ComPortList.SelectedItem.ToString();
try
{
    MEAHeadstageProgram.cereal.Open();
}
catch
{
    println("Error connecting to " + MEAHeadstageProgram.cereal.PortName);
    ConnectionStatusText.Text = "Failed to open Com Port!";
    return;
}
MEAHeadstageProgram.cereal.DataReceived += new SerialDataReceivedEventHandler(MEAHeadstageProgram.Cereal_DataReceived);

C#: 
internal static void Cereal_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
    try
    {
        RxBuffer = cereal.ReadLine() + Environment.NewLine;
        F.OnReceive();
    }
    catch
    {
        F.println("Error reading from " + MEAHeadstageProgram.cereal.PortName);
        return;
    }
}

C#: 
internal void OnReceive()
{
    if (MEAHeadstageProgram.RxBuffer.Length > 0)
    {
        string str = MEAHeadstageProgram.RxBuffer.TrimEnd('\r', '\n');

        // parse all stuff coming from teensy
        if (str.StartsWith("DATA "))
        {
            if (spikePlotStatus == SpikePlotStatus.Play || spikePlotStatus == SpikePlotStatus.Record)
            {
                // println("encoding recieved");
                Stopwatch watch = Stopwatch.StartNew();

                string encoded_content = str.Substring(5);

                if (spikePlotStatus == SpikePlotStatus.Record)
                {
                    recordingWriter.WriteLine(encoded_content);
                }

                if (encoded_content.Length == 176)
                {
                    mea_data_parser.base64_decode(encoded_content);

                    if (display_downsample_factor_tracker == 0)
                    {
                        for (int i = 0; i < 64; i++)
                        {
                            channels_data[i][stream_position] = mea_data_parser.calculated_channel_data[i];
                            timestamp_data[stream_position] = mea_data_parser.timestamp;

                        }
                        //println(mea_data_parser.timestamp.ToString());


                        stream_position++;

                        if (stream_position == graph_sample_size)
                        {
                            stream_position = 0;
                        }
                    }

                    display_downsample_factor_tracker++;

                    if (display_downsample_factor_tracker == display_downsample_factor)
                    {
                        display_downsample_factor_tracker = 0;
                    }
                }
                else
                {
                    println("! Encoded content length error: " + encoded_content.Length.ToString());
                }

                watch.Stop();
                long elapsedMicroseconds = watch.ElapsedTicks * 1_000_000L / Stopwatch.Frequency;
                if (elapsedMicroseconds != GPUCycleTimeTracker)
                {
                    GPUCycleTimeTracker = elapsedMicroseconds;
                    GPUCycleText.Text = "GPU Cycle: " + GPUCycleTimeTracker.ToString() + " us, max " + (1000.0 / GPUCycleTimeTracker).ToString("0.##") + " kHz";
                }
            }
            else
            {
                GPUCycleText.Text = "GPU Cycle: _ us, max _ kHz";
            }

            return;
        }
        if (str.StartsWith("CYCLE "))
        {
            string cycle_text = str.Substring(6);
            TeensyCycleText.Text = cycle_text;
            println(cycle_text);
            return;
        }
        if (str.StartsWith("STIMMED"))
        {
            // EnableChannelStim_CheckBox.Enabled = true;
            StimParamPannel.Enabled = true;
            println("Stim Completed");
            return;
        }

        println("↘️ " + str);
    }
}
 
What happens if the baud rate is changed? The Teensy doesn't care - but the PC may be throttling it - somewhere I saw that being the case once - or so it seemed.

MEAHeadstageProgram.cereal.BaudRate = 1152000;

Setting this to 6,000,000 instead of 1.15M ?
 
bigdlin said:
Doesn't seem to affect it... still bad
Click to expand...
Bummer - Opps baud ... though 6M is only 600K chars/sec - and 12M is slow usb and T_4 is 40X faster. Maybe try again asking for 60 Mbaud?

There was a 'C' exe used when testing unrestrained Teensy to PC comms - and Windows buffer store/deliver seems the slow part. But that was over 8M chars/sec with 250K lines of 32 bytes/sec. So it is possible - and that exceeds the needs there.
 
Same i don't think C# baud rate does anything. I've tried to change it to 75 and still got the same thing. Yeah, I have read that doc already. The only thing done differently is using a teensy-specific com port scanner, but after it's connected, we are doing identical tasks.
 
Can confirm the baud rate setting has no effect.

Actual communication happens as USB native speed with end-to-end flow control. Conceptually the flow control is similar to traditional hardware serial RTS/CTS, but it's built into the low-level USB protocol and always active. Like with traditional RTS/CTS flow control, the overall speed you can achieve is at most the native wire speed, but will automatically slow down without data loss if either or both sides actually run slower.

Saying baud rate setting has no effect isn't technically true. The baud rate you use on the PC side is communicated to Teensy, but not actually used unless your program uses Serial.baud() to read it. Even then, it's not used for anything on USB. Typically your program would use Serial.baud() is you were making a USB-serial converter where your program copies all data back and forth to a hardware serial port (Teensy 4.1 has 8 of them). In that case, Serial.baud() is quite useful for configuring the hardware serial port, since your program on Teensy can see what setting the PC software wanted. Because the USB protocol has end-to-end flow control built in, unless the PC side is incredibly slow the hardware serial baud rate would be the pacing limit for that sort of use.

None of that applies in this type of application. Here the baud rate setting has no effect. Don't waste your time with it. The performance problems you are seeing are due the effect of flow control. Its purpose is to slow down the communication to prevent data loss when the receiving side is unable to digest the incoming data fast enough.
 
I personally have not used C#. I really have no idea whether it can be used to receive data fast enough. But I'm going to go out on a limb with comments about the code in msg #3.... but please understand my comments are about communication in general based mostly on experience with C and C++ using native WIN32 API.

Your C# code looks like it's using some sort of callback which C# provides. Maybe? I see you have something called SafeSerialPort with syntax that kinda looks like it's built on top of SafeSerial, which a google search turned up on Mircosoft's website. But I don't see any mention of callbacks or OnReceive.

Guessing and troubleshooting performance bottlenecks is tough even in a very familiar language and experience with APIs, so this guesswork comes with a huge caveat. With that in mind, I would imagine several possible causes of the performance issue, in the order of importance I would blindly guess without knowing pretty much anything of C#.

1: Your OnReceive code seems to be doing quite a lot of processing. Maybe spending too much time here hurts your ability to receive more incoming data? I see several API calls and some light data manipulation, but I have no idea how expensive this stuff is in C#.

2: Your OnReceive function might be causing GUI events that end up repainting. This is horribly inefficient on any OS, but particularly bad with Windows. Or at least has been in my experience, because Windows seems to do GUI updates synchronously. Maybe C# is newer and Microsort has improved how things are done, but my cynical side has doubts. Ideally, you would want to be sure your code causes GUI events that ultimately lead to a screen repaint to no more than whatever your monitor's vertical refresh rate is. A function like OnReceive could run many thousands of times per second during high speed data flow. But the Windows GUI will struggle to process 120 or even 60 repaint-causing events per second. I see you have a println() call near the end, which I would be suspect. You should really consider how many GUI repaints per second everything you call might cause to happen.

3: Maybe SafeSerialPort adds too much overhead. Perhaps if it's parsing incoming bytes into wide characters and looking for line breaks. That sort of code ought to be fast, but sometimes things designed in these modern environments end up having surprising overhead for things you would imagine doing quickly in native C code. You might need to use (unsafe) SerialPort for more direct access to the WIN32 API.

4: Perhaps all this stuff is being run in an event-based system (designed around Windows GUI) where you're always going to suffer too much overhead or latency? You might need to create a dedicated thread which just receives the incoming data and somehow hands it off to the rest of your program. If #2 is an issue, this is probably where you would gather and pre-process data so you can deliver it in smaller number of larger chunks so you don't risk flooding the GUI with too many expensive repaint-causing events. Ultimately the path to successfully processing high speed data is to do it in large enough chuncks that the fixed overhead of API calls, context switches, function entry/exit, etc... gets amortized over many bytes.

5: C# might just not be up to this task. I suspect this is unlikely, but I can say with certainty it can be done using C / C++ using native WIN32 APIs.

Again, I really don't know anything about C#. But I tried to look at your C# code anyway. Please remember this is based on my experience with C / C++ and native WIN32 APIs. Hopefully it helps?
 
Couple of years ago I did a lot of experiments with high speed data transfer over usb serial. It turned out that on Win10 you can stably transfer some 10MByte/sec from a Teensy to a dotNet (C#) application.

However, there is a known bug in the Windows CDC driver which silently drops data if you have the teensy freely spitting out large data onto the bus. One can work around this by throttling the speed. E.g., send smaller (e.g. 10kB) blocks and delay between sending the blocks. If you calculate your delay such that you end up with some 5MByte/sec. overall data rate you get a stable transmission.

It worked much better when the teensy only sends on demand. I.e., the host requests a large block (say 100kB) after which the Teensy sends exactly the requested amount of data. If the host got it, it requests another block and so on. In this case throttling was not necessary and I was able to maintain transfer rates of some 13MByte/sec.

@bigdlin I should have a simple test program somewhere. If you are interested I can try to find it and see if it still works...

Here one of the threads dealing with the issue https://forum.pjrc.com/index.php?threads/serial-write-xmit-dropping-data.68382/page-2#post-290663
 
Last edited:
You must log in or register to reply here.
Back
Top