Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 11 of 11

Thread: Serial1 output & flushing.

  1. #1
    Senior Member jim lee's Avatar
    Join Date
    Dec 2016
    Location
    Anacortes, WA
    Posts
    156

    Serial1 output & flushing.

    I'm having issues that I suspect may be my Serial1 is getting stopped before its sent out all of its data. I sent the output to the monitor and I get it all, send it to Serial1 and I get 63 bytes.

    So, to help my debugging.. Is there any way/call I can do to block myself 'till the outgoing text is all written out? or read how much is left? Anything?

    Thanks!

    -jimlee

  2. #2
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,311
    You talk about an issue with sending out, then lead on to say you have an issue receiving. Theres 2 things we need to deal with and without knowing your expectations or code example it's hard to say well it could be this, it could be that, we don't know exactly until you provide code and more details.

    You may see how many bytes are available using Serial1.available(), and I am guessing you are writing more than 64 bytes to Serial1 which causes overflow and tossed characters. The buffer is 64 bytes.

  3. #3
    Senior Member jim lee's Avatar
    Join Date
    Dec 2016
    Location
    Anacortes, WA
    Posts
    156
    Well sure. In one end I'm sending a bunch of stuff. In the other end of the hose I'm seeing what comes out. If the hose is connected to the monitor on the development computer, I get what I expect. When its switched to the other teensy, I don't.

    But, I'm not asking people to fix my code. I just want to see if there's a way to hold 'till I know everything has been written out, so I can figure out what I'm doing wrong.

    -jim lee

  4. #4
    Junior Member
    Join Date
    Aug 2018
    Location
    Brisbane, Australia
    Posts
    15
    You can track the amount of send buffer available by calling int availableForWrite(void)

  5. #5
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,311
    You also have to make sure the other teensy (which is another detail you are using 2 teensy's, not mentioned earlier...) has no code blocking the reads of the Serial1 buffer which you should be handling there. Even if your within the limits of availableForWrite it won't guarentee the other end processes the data properly "in time". So with 2 teensies you should be able to provide sketch examples of both so we can see if your code has any flaws that we can correct.

    For the most part, not many people handle the write side (most do not use availableForWrite). The important thing is to handle the receive side properly.

  6. #6
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    21,808
    I'm pretty sure you're asking the wrong question, but I'll try to answer anyway.

    I just want to see if there's a way to hold 'till I know everything has been written out
    This is what Serial1.flush() does. It waits until everything has been transmitted.

    You can also use Serial.availableForWrite(), which tells you how much space is ready in the transmit buffer. This is not exactly the same as Serial1.flush(). If you call availableForWrite() before sending anything, then after sending keep calling it until you see the space as returned to its previous amount, that condition occurs while the last few bytes are still transmitting. Exactly how many depends on which Teensy you use, as they have different size FIFOs, or no FIFO at all on Teensy LC, but still a double-buffer transmit register.

    If you *really* want to see the timing, the best way is to use digitalWriteFast() and use a logic analyzer or oscilloscope to see the moment that pin changes relative to the actual serial data.

  7. #7
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    11,488
    Also important is what Teensy is in use? What is on the other end of Serial1?

    As noted without a (short) sketch and repro steps any answers would be guesses, as this would normally be expected to work. Many of us have connected two or more Teensy's together for extensive Serial UART testing of debug or other function testing.

  8. #8
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    21,808
    However, the problem you're seeing is very likely an issue with overflowing the receive buffer. Focusing your attention on the transmitter timing is probably just a distraction.

    But first, you should understand that USB virtual serial has end-to-end flow control built into the USB protocol. When you transmit a large amount of data, it's written into buffers on the Teensy side. Each buffer is only sent when the PC is able to receive it. If the PC isn't ready, the data remains sitting on the Teensy side. USB is a complex protocol, but for this case the thing you need to understand is the USB protocol fundamentally includes flow control negotiation between the PC and Teensy, so you can't get data loss by sending too fast. Because Serial.print() and Serial.write() wait when all buffers are full, the actual end-to-end speed is only as fast as the software on both side can run. This flow control is built into the USB protocol at the lowest level. It can't be turned off. It's always how USB virtual serial works (but again, with some caveats I've glossed over to keep this message shorter).

    Real hardware serial works the opposite way, unless you're using the optional RTS/CTS hardware flow control. When you transmit data, it's also written into a buffer on the Teensy side, which allows your program to keep running without waiting (unless the buffer fills up). But unlike USB, and without flow control signals, the data immediately begins transmitting, paced only by the baud rate. Teensy can't know whether the other side is ready or able to receive. The voltage on the TX1 pin simply toggles with the bits you wrote. There is no way to assure the other side really will really will receive the byte and store it into a buffer.

    We see this serial data loss problem over and over on this forum. You transmit bytes, but they don't get received on the other side. It's almost always because the receive buffer filled up and the receiving Teensy couldn't accept more data as it arrived at the RX1 pin, because the receive buffer had no more space at that moment.

    Usually the problem is poorly designed receive-side code which calls Serial1.available(), but doesn't actually do Serial1.read() until some large amount is reported. We can't see your code to know whether you're doing anything like that, but it is a common issue. The other less common problem is doing some lengthy operation, or just a delay, without checking Serial1.availalbe() and calling Serial1.read() to get the received bytes quickly out of the buffer, so you have room for more to arrive. Usually this is an issue only at high baud rates. A 3rd problem we see here involves certain libraries which block interrupts, disrupting the serial receive no matter what you do with Serial1.availalbe() and Serial1.read(). Adafruit_NeoPixel is the most common problem, and switching to WS2812Serial is the solution.

    Of course this is all just a lot of guesswork, and a lengthy message to explain the common problems... when it could have been a much shorter message focusing on the specific problem you're probably facing, had you been clearer about what you're doing and shown us the code. But hopefully this long and very general explanation helps anyway?
    Last edited by PaulStoffregen; 03-28-2020 at 07:53 AM.

  9. #9
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    21,808
    Often the "easy" solution is to just edit the serial code to make the receive buffer much larger. Exactly which file to edit depends on which Teensy model you're using.

  10. #10
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,311
    Also note that editing the core buffer size removes portability of your code on different platforms, and upgrading teensyduino returns it to stock value of 64 so you'd have to edit again after an update. Teensy is pretty fast at handling the data provided your code isn't delaying the throughput

  11. #11
    Senior Member jim lee's Avatar
    Join Date
    Dec 2016
    Location
    Anacortes, WA
    Posts
    156
    Guys really, I think you nailed it on the head that the receive is not up to the job. I just wanted to confirm that the Serial.flush() was doing what I thought it was doing, blocking 'till it's all gone. And confirm my transmit is working ok. What you said about the USB being different than the teensy totally sheds new light on this. The issue I'm having is between two teensy 3.2s. All my code between the two follows a strict protocol of first byte is length of packet coming. Its pretty bullet proof. This is one -tiny- hack I added that sends an un-acked packet out, then sits back for a flood of text of unknown length. It looks like, at 63 bytes it stops for a breather. My code goes off and does a 5,000 year long screen refresh (Its a little handheld serial terminal emulator, see below) and by the time it comes back.. The stuff is gone and the port has been closed.

    Simple answer is, I need to redesign how I'm doing this.

    Here's the emulator showing its 63 bytes of return data.

    Click image for larger version. 

Name:	sTerm.jpg 
Views:	7 
Size:	309.1 KB 
ID:	19519

    And really, Thank you all for the info. I know I keep falling into this serial thing. Then I forget if flush() blocks? Or does it just clear out the buffers?

    And no, I have no desire to mess with the core buffer size or anything else down there. One of the main reasons I like these Teensys, is that as far as I can tell, they are pretty rock solid reliable.

    -jim lee

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •