Teensy 4.0 Serial.available() stops incrementing

Status
Not open for further replies.

noisymime

Active member
OK, I really hope I'm doing something dumb, but I'm having all sorts of trouble with some of the serial functions under Teensy 4.

The issue I'm seeing is that the Serial.available() stops updating after the first call to it, unless there is a read performed. Here's an example sketch:
Code:
void setup() {
  // initialize serial communication at 9600 bits per second:
  Serial.begin(9600);
}

// the loop routine runs over and over again forever:
void loop() {
  Serial.println(Serial.available());
  if(Serial.available() > 10)
  {
    while(Serial.available())
    {
      Serial.read();
    }
  }
  delay(1000);        // delay in between reads for stability
}

Basically the sketch should print out the number of bytes on the serial RX buffer. If the number on the buffer is greater than 10, it'll read all the values back out from the buffer.

However, if I run that sketch I start seeing 0 on the monitor as expected. If I send a byte of data, the value goes up to 1, again as you'd expect. But if I send another byte, it still stays as 1. Send another byte and its still 1 etc.

If I reset the board and this time instead send say 5 bytes the first time, it'll start showing 5, but just as before if I send more bytes, it still shows as 5.

If I send 11 bytes initially, then it'll print 11, then back to 0, which is correct.

Am I missing something obvious about how the Serial.available() function works or should it keep going up with each new byte received (Even if there isn't a Serial.read() call)?
 
Seeing the same behavior on a T4 here.

I have seen signs of this error before in Beta but never pinned it down while prompting for user input - TyComm would indicate COMM errors as shown below. @mjs513 and/or @KurtE once noted this IIRC during Beta.

Modified the sketch to read ONE char after 10 secs when the .available() > 1 and even reading one char doesn't update the count to include new characters passed in - then repeats after another 2 seconds.
>> send 9 chars '12345678\n'
>> Send more chars and they are not included in the count

Restarted IDE and new FirstTrySerMon update and same result - only no errors in the IDE console.

Added cnt2nd - that will flush with Serial.read() after repeated waits - it is not getting buffered chars on TyComm because of the write error - but teensy_sermon does pull those added chars when cnt2nd triggers.

And if I keep passing in characters I get a red flag Error in TyCommander:
Code:
[send@6052840-Teensy] I/O error while writing to '\\.\COM33'
[send@6052840-Teensy] Timed out while writing to '\\.\COM33'
[send@6052840-Teensy] Timed out while writing to '\\.\COM33'
Board '6052840-Teensy' is busy on task 'send@6052840-Teensy'
Board '6052840-Teensy' is busy on task 'send@6052840-Teensy'
[send@6052840-Teensy] Timed out while writing to '\\.\COM33'

Here is the updated sketch to demonstrate:
Code:
void setup() {
	// put your setup code here, to run once:
	Serial.begin(115200);
	while (!Serial && millis() < 4000 );
	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
}

uint32_t cnt = 0;
uint32_t cnt2nd = 0;
void loop() {
	Serial.println(Serial.available());
	cnt++;
	if (Serial.available() > 10)
	{
		cnt = 0;
		while (Serial.available())
		{
			Serial.read();
		}
	}

	if (Serial.available() >= 1 && cnt > 10) {
		if (Serial.available() > 1 ) {
			Serial.read();
			cnt = 8;
		}
		cnt2nd++;
		if ( cnt2nd > 5 ) {
			cnt2nd = 0;
			while (Serial.available())
			{
				Serial.read();
			}

		}
	}
	delay(1000);        // delay in between reads for stability
}

*yes, a 'spec' bug - it should not cnt++ when no chars yet .available() - but it just reads too soon and shows the problem with less wait.
 
Last edited:
In all these T4 serial-usb discussion, one should note that in addition to Paul's Sermon speed work, the USB implementation is 'completely' different and most likely not complete in a T4 compared to T3.
[Rant] honestly, Paul should not waste his time speeding up Sermon, but concentrate on T4 USB implementation (e.g. other device than Serial). I never had problems to slow down Serial such that I can read the text[\Rant]
 
SerMon speedup is handy - crashing the IDE and losing work is ugly - T4 too easy to overload or overwork a PC. I use TyCommander - it doesn't fail but only runs half as fast and uses double the CPU % which makes doing other things fitful.

Also much of the coming USB work will make it faster and being able to test it and have it run at speed is where troubles may show up.

@Gadget999 - not sure that applies? If the chars are READ there is no trouble - but when waiting for a given count of chars to arrive and reading is delayed - the problem here is the count of chars will never change with incoming characters.
 
A little more playing and I can see that there's definitely an issue here with subsequent serial data being sent. It's not just that Serial.available() isn't being updated, it's that the subsequent data isn't being added to the rx buffer at all.

Within usb_serial.c when the first set of bytes is sent I can see the rx_event() function being called as you'd expect. This function updates the rx_count[index] and rx_index[index] values to store the current place in the buffer. However, no matter how much data is sent, that rx_event() function is not called again until all the data has been read() out of the buffer from the first data that was sent.

I've been playing around to see why that function isn't being called for subsequent bytes, but haven't had much luck yet.
 
A little more playing and I can see that there's definitely an issue here with subsequent serial data being sent. It's not just that Serial.available() isn't being updated, it's that the subsequent data isn't being added to the rx buffer at all.

Within usb_serial.c when the first set of bytes is sent I can see the rx_event() function being called as you'd expect. This function updates the rx_count[index] and rx_index[index] values to store the current place in the buffer. However, no matter how much data is sent, that rx_event() function is not called again until all the data has been read() out of the buffer from the first data that was sent.

I've been playing around to see why that function isn't being called for subsequent bytes, but haven't had much luck yet.

Indeed some issue here for @PaulStoffregen. I see he has visited this thread - hopefully it is on his list.

It was ODD that teensy_sermon did seem to get characters pushed in after the buffered characters and as noted TyComm detected and returned write errors. The t_sermon must be holding the buffer to send when the USB resumes normal cycling after the buffer gets emptied in T4. Those two serial monitors must be using the USB state/control feedback differently - perhaps TyComm throws Error when t_sermon goes to retry/wait? Either way incoming data seems to be blocked/ignored on the T4 side.
 
After more playing, I've managed to confirm a few more things around this, but not exactly where the main problem is coming from. As best as I can tell, the serial RX interrupt is being disabled after the initial data is received and only gets turned back on after the data has been read back out, which would explain the issue.

Just for clarity, this is not related at all to the serial monitor issues at all, it happens with the stock Arduino IDE monitor as well as 3rd party ones. I'm actually trying to interface with a totally separate application that has a serial interface and works fine with AVR and Teensy 3.5/3.6 boards, but is dependant on multiple data bursts being sent before a read() is called, so its obviously failing with the current T4 usb serial.

I'll dig out a 3v3 serial board over the weekend and try one of the native UARTs to see whether they have the same issue or not.
 
Just to confirm back, using a 3v3 USB serial adapter and Serial1 on the Teensy 4.0, everything works perfectly, so this problem is definitely within the USB serial implementation.
 
Just to confirm back, using a 3v3 USB serial adapter and Serial1 on the Teensy 4.0, everything works perfectly, so this problem is definitely within the USB serial implementation.

Not surprising - it got to a very functional state for most use cases - it got good enough for Beta use and without specific use case issues beyond added development nothing like this showed up, as noted I did see signs of trouble but nothing blocking or reproducible to make issue of - like this. Further work is staged on Paul's schedule with a major update to move USB to DMA and also add in all he other Serial types the T_3.x's present. I've made enough notes this should be 'on the list' - though no direct notes to that effect yet with other tasks taking priority.

I did just get confirmation of taking a Pull request I made to cores for another issue - so as usual Paul is busily working … and github gave me notice :)
 
Looking at this one now. If anyone remembers back to the lengthy T4 beta test, for a long time we had only USB serial transmit. I wrote pretty simple receive code (so it would be reliable and stable), with the intention to someday improve it and make it more efficient. Looks like that time has come...

Currently (version 1.48) the USB serial receive code on Teensy 4.0 is holding only a single USB packet at once. The receiving code can't receive another until you've read the data of the previously received packet. If your PC sends a packet with only 1 byte, then it can't receive the next packet until you've read than 1. If the packet has 100 bytes, you get 100 from Serial.available(), but again the next packet can't be received until you've read all 100 bytes.

Did I mention the idea was to keep this as simple as possible? Yeah, it's terribly inefficient. I'll write something much better this weekend. Will need as much help as possible with testing!

But at least you can hopefully see what Serial.available() isn't incrementing. It's holding only 1 packet, which can be anywhere from 1 to 64 bytes (up to 512 bytes if using the latest code from github) and can't receive another until you're read all bytes of the one packet it has buffered.
 
<edit> : Cross post - got back to this without refresh ... indeed looks like the K.I.S.S. of death here :)

Paul - this one is not fixed with the USB Github update from earlier - the available count sticks at the first batch of incoming - added incoming bytes are not showing added to count.

Run post #2 code - send over some chars - shows incoming count - send more chars and count does not change.

The p#2 code gets bored waiting and starts pulling off what is there and that works.

However: code waiting for 100 chars when they arrive in two sets of 50 would never complete as the .available() would show 50 with the first batch and not change with added characters.
 
… cross posted … Understood … edit above … finally installing VStudio 2019 to replace 2017 I removed … me and my slow interWebs is distracted with a 2.6 GB download on a 650 KB/sec downlink :(

… wanted to confirm is was still as repro - wasn't sure what was in the github USB updates from last month …
 
Paul - tested this code against T_3.6 to work as follows:

> Run Sketch : It will wait to read any chars until Serial.available() gets three unique non-zero values - that will be increasing with new chars reported.
--> Send a group of chars : Will show new char counts
--> Send a group of chars : Will show new char counts
--> Send a group of chars :: Will READ all and print the number of chars read
>> If it gets bored waiting over 10 seconds for new chars when above conditions not met - it will prompt for further input - which is what T4 would do now

Code:
void setup() {
	Serial.begin(115200);
	while (!Serial && millis() < 4000 );
	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
	Serial.printf( "\nWaiting for three incoming Avail update counts \n\n" );
}

int lastAvail = 0;
int lastAvail2 = 0;
elapsedMillis someWait;
void loop() {
	if ( someWait > 10000 ) {
		if ( Serial.available() ) {
			someWait = 0;
			Serial.printf( "Waiting for three incoming Avail update counts %u : %u : %u\n", Serial.available(), lastAvail, lastAvail2 );
		}
	}
	if ( Serial.available() != lastAvail && lastAvail != lastAvail2 && 0 != lastAvail2 )
	{
		int ii = 0;
		Serial.printf( "\tGot three incoming Avail update counts %u : %u : %u", Serial.available(), lastAvail, lastAvail2 );
		while (Serial.available())
		{
			Serial.read();
			ii++;
		}
		Serial.printf( "\t\tBytes read %u\n", ii);
		lastAvail2 = lastAvail = 0;
	}

	if ( Serial.available() != lastAvail )
	{
		someWait = 0;
		lastAvail2 = lastAvail;
		lastAvail = Serial.available();
		Serial.printf( "Update to incoming Avail counts %u : %u : %u\n", Serial.available(), lastAvail, lastAvail2 );
	}
}

It looks like this where waiting prints after some chars received but it is still waiting - perhaps 10 seconds:
Code:
T:\tCode\T4\SerAvailTest\SerAvailTest.ino Nov 29 2019 23:31:48

Waiting for three incoming Avail update counts 

Update to incoming Avail counts 18 : 18 : 0
Waiting for three incoming Avail update counts 18 : 18 : 0
Update to incoming Avail counts 36 : 36 : 18
	[B]Got three incoming Avail update counts 54 : 36 : 18		Bytes read 54[/B]

Running on current code with T4 it shows this even though char group after first was sent:
Code:
T:\tCode\T4\SerAvailTest\SerAvailTest.ino Nov 29 2019 23:45:07

Waiting for three incoming Avail update counts 

Update to incoming Avail counts 5 : 5 : 0
Waiting for three incoming Avail update counts 5 : 5 : 0
Waiting for three incoming Avail update counts 5 : 5 : 0
Waiting for three incoming Avail update counts 5 : 5 : 0
 
I am holding off migrating from t3.2 to 4.0 because of this.

Can i suggest a simple option to ignore all the held up data and process the latest sent

Ie you have read data 1 of 100 and then go straight to 100 if there is a delay

It will ensure the sending program and t4 are both synchronised in time
 
Can i suggest a simple option to ignore all the held up data and process the latest sent

That would break a huge amount of code which depends on Serial to work properly without losing incoming data while the sketch/program is busy doing other work!


I am holding off migrating from t3.2 to 4.0 because of this.

In a matter of days I'm confident we'll have a beta which solves this problem, and very likely a stable release before January.
 
I can see how that may not suit everyones needs.

Great news on the january release. I will look to get my boards t4 compatible :)
 
@PaulStoffregen - That will be great!

@Gadget999 - In the mean time, you might be able to play around to see if you can avoid the issue...

And again I don't know if it would work for you or not. At different times on different platforms, I have had some issues with Serial.available (or maybe SerialX.available...)
And besides why do two calls when you can do one...

That is if you have code like:
Code:
while(Serial.available()) {
    mychar = Serial.read();
...
}

You can instead do something like:
Code:
int ch;
while ((ch= Serial.read()) != -1) {
    // process the character
}
 
Paul Noted on other fixed thread about 'this' thread:

...
The USB serial receive code is still far from optimal. This is the thread where I'd like to discuss the coming receive code improvements.
...

With that other issue fixed based on post #12 by Paul
Code:
I'll write something much better this weekend. Will need as much help as possible with testing!

The Post #16 code above works well on T_3.6 as it should work with functional T4 Multi_packet receive as a baseline.
 
Paul - I saw github Update - it passes the above post #16 code test it seems?

Just room for TWO large buffers so far?

Update to incoming Avail counts 11 : 11 : 0
Update to incoming Avail counts 22 : 22 : 11
Got three incoming Avail update counts 33 : 22 : 11 Bytes read 33

Though it doesn't seem to like it if the send is a large 2800 bytes?
<EDIT>: Will change sketch to READ when the .available() is over 'some' value - will start with 512?

Or after sending two sets of 280 bytes - it isn't ready for even 1 more char?
Waiting for three incoming Avail update counts

Update to incoming Avail counts 280 : 280 : 0
Update to incoming Avail counts 560 : 560 : 280
Waiting for three incoming Avail update counts 560 : 560 : 280
Waiting for three incoming Avail update counts 560 : 560 : 280

And it does work for 1 char {blank enter} and then 280 chars twice - but then no longer responsive:
Waiting for three incoming Avail update counts

Update to incoming Avail counts 1 : 1 : 0
Update to incoming Avail counts 281 : 281 : 1
Got three incoming Avail update counts 561 : 281 : 1 Bytes read 561

Though this string (65 chars+Enter): "123456789012345678901234567890123456789012345 6789 01234567890abc" does work:
Code:
Waiting for three incoming Avail update counts 

Update to incoming Avail counts 66 : 66 : 0
Update to incoming Avail counts 132 : 132 : 66
	Got three incoming Avail update counts 198 : 132 : 66		Bytes read 198
Update to incoming Avail counts 66 : 66 : 0
Update to incoming Avail counts 132 : 132 : 66
Update to incoming Avail counts 198 : 198 : 132
	Got three incoming Avail update counts 264 : 198 : 132		Bytes read 264

Though something ODD in that last - maybe a bug in above sketch?
Number look unique as expected - but it doesn't trigger and the 66 got bumped off ...
Waiting for three incoming Avail update counts 66 : 0 : 0
Update to incoming Avail counts 66 : 66 : 0
Update to incoming Avail counts 132 : 132 : 66
Update to incoming Avail counts 198 : 198 : 132
Waiting for three incoming Avail update counts 198 : 198 : 132
 
Last edited:
Thanks for testing. But it is still very much a work in progress. I can tell you it will fail if you transmit a single large write of more than 1024 bytes, followed by 2 small writes. That's the problem I'm working to fix, but it's looking like a tough issue deeper in usb.c.

Just room for TWO large buffers so far?

It's currently allocating 3 buffers, by this line:

Code:
#define RX_NUM  3

Whether all 3 really are used properly is another question at this early state of testing...
 
Thanks for testing. But it is still very much a work in progress. I can tell you it will fail if you transmit a single large write of more than 1024 bytes, followed by 2 small writes. That's the problem I'm working to fix, but it's looking like a tough issue deeper in usb.c.



Whether all 3 really are used properly is another question at this early state of testing...

It is MUCH better in the right direction.

Assumed it was a WIP - if it was that easy you would have it done already. Saw the github update and wanted to see if the sketch was a decent test ...

Did some edits on the sketch - one main read of .available per loop for testing - and cascaded the 3 IF's through Else's as it was getting crossed up with untimely USB count changes.

For ref here are two cases that fail - 3 reads complete - then T4 needs reset as it won't continue. They were not three full 512 B sends:
Code:
Update to incoming Avail counts 181 : 181 : 0
Update to incoming Avail counts 362 : 362 : 181
	Got three incoming Avail update counts 543 : 362 : 181		Bytes read 543
T:\tCode\T4\SerAvailTest\SerAvailTest.ino Nov 30 2019 18:58:44 << THIS SHOWS RESET

and
Code:
Update to incoming Avail counts 61 : 61 : 0
Update to incoming Avail counts 341 : 341 : 61
	Got three incoming Avail update counts 621 : 341 : 61		Bytes read 621

T:\tCode\T4\SerAvailTest\SerAvailTest.ino Nov 30 2019 18:58:44 << THIS SHOWS RESET

Code:
void setup() {
	// put your setup code here, to run once:
	Serial.begin(115200);
	while (!Serial && millis() < 4000 );
	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
	Serial.printf( "\nWaiting for three incoming Avail update counts \n\n" );
}

int lastAvail = 0;
int lastAvail2 = 0;
elapsedMillis someWait;
void loop() {
	int myAvail = Serial.available();
	if ( myAvail>512 || (myAvail != lastAvail && lastAvail != lastAvail2 && 0 != lastAvail2 ) )
	{
		int ii = 0;
		Serial.printf( "\tGot three incoming Avail update counts %u : %u : %u", myAvail, lastAvail, lastAvail2 );
		while (Serial.available())
		{
			Serial.read();
			ii++;
		}
		Serial.printf( "\t\tBytes read %u\n", ii);
		lastAvail2 = lastAvail = 0;
	}
	else if ( myAvail != lastAvail )
	{
		someWait = 0;
		lastAvail2 = lastAvail;
		lastAvail = myAvail;
		Serial.printf( "Update to incoming Avail counts %u : %u : %u\n", myAvail, lastAvail, lastAvail2 );
	}
	else if ( someWait > 10000 ) {
		if ( myAvail ) {
			someWait = 0;
			Serial.printf( "Waiting for three incoming Avail update counts %u : %u : %u\n", myAvail, lastAvail, lastAvail2 );
		}
	}
}
 
Status
Not open for further replies.
Back
Top