Teensy 4.x Serial ports hanging?

[lpaolini]

I'm using Teensy 4.x Serial ports for driving LED strips via WS2812Serial, either directly or indirectly via FastLED and they seem to freeze at random times (all LED strips at the same time). This happens especially while flashing new code, but also while running.

The most interesting part is that it seems to depend on code, somehow. It happened several times that a small (and unrelated) change made it unstable (freezing) while it was perfectly stable just before that change.

The only way for restoring functionality is to power down.

Is there a way for manually ensuring all Serial ports are properly reset?

Should you need to look at my code, here it is: https://github.com/lpaolini/Striptease

Thanks,
Luca

 

[PaulStoffregen]

Can you give me a small self-contained program (using only libs Teensyduino installs) which I can copy into Arduino (not PlatformIO) to reproduce the problem?

To specifically answer your question:

Is there a way for manually ensuring all Serial ports are properly reset?

Serial1.begin(), Serial2.begin(), etc are supposed to fully initialize the serial port hardware. If using WS2812Serial, that code is built into the WS2812Serial library, so you should not add Serial1.begin().

It happened several times that a small (and unrelated) change made it unstable (freezing) while it was perfectly stable just before that change.

We have seen these sorts of problems over and over on this forum. Almost always the issue turns out to be a buffer overflow. These bugs are very hard to troubleshoot, partly because even small changes to your program can alter the layout of items in memory, partly because a bug which overwrites other variables tends to manifest problems *much* later when other code actually uses those corrupted variables.

 

[lpaolini]

Hi Paul,

Many thanks for your reply.

Can you give me a small self-contained program (using only libs Teensyduino installs) which I can copy into Arduino (not PlatformIO) to reproduce the problem?

I'll try to shrink it down and send it over to you the next time it happens.

Serial1.begin(), Serial2.begin(), etc are supposed to fully initialize the serial port hardware. If using WS2812Serial, that code is built into the WS2812Serial library, so you should not add Serial1.begin().

Ok.

We have seen these sorts of problems over and over on this forum. Almost always the issue turns out to be a buffer overflow. These bugs are very hard to troubleshoot, partly because even small changes to your program can alter the layout of items in memory, partly because a bug which overwrites other variables tends to manifest problems *much* later when other code actually uses those corrupted variables.

Yeah, exactly what I was thinking.
I'll go through my code again, but I'm not very positive about finding something as it's pretty "high level" code.

Thanks again.

 

[lpaolini]

It happened again, after many hours.

Right now all my LED strips, connected to Teensy 4.1 Serial 1, 3, 4 and 7 and frozen, but the program is running.
I can use the serial monitor or the infrared remote to interact with it and everything looks fine, except for the LED strips.

I have tried re-uploaded the same code: it succeeds but nothing changes.
What I'm thinking is that Serial ports are now in a bad state.

Is this possible? Is the reset triggered after uploading any different from a hard reboot?
I'm really confused. :-/

 

[PaulStoffregen]

Is the reset triggered after uploading any different from a hard reboot?

After uploading, the bootloader chip uses the hardware reset signal to fully reset the main processor. The serial port hardware is supposed to be fully reset.

Let me answer now with another question. How could you know whether the problem is Teensy or your LEDs?

Maybe try wiring the LEDs to separate power (USB power might be simplest), so you can power cycle Teensy without also power cycling the LEDs.

 

[lpaolini]

After uploading, the bootloader chip uses the hardware reset signal to fully reset the main processor. The serial port hardware is supposed to be fully reset.

Ok.

Let me answer now with another question. How could you know whether the problem is Teensy or your LEDs?

I don't know for sure, I'm just guessing.
Between the serial ports and the strips there's a single 8-port level converter (74HCT245) followed by a 100ohm resistor for each port.
What makes me think of the serial ports is the fact that all strips freeze at the same time.
The only other reason I can think of is a faulty 74HCT245, which is quite unlikely I guess.

Maybe try wiring the LEDs to separate power (USB power might be simplest), so you can power cycle Teensy without also power cycling the LEDs.

Sure, I'll try that. I can disconnect the main board (holding the Teensy, the Audio Board and level shifter) while keeping the LED strips powered, then reconnect the board and see what happens.

 

[lpaolini]

Maybe try wiring the LEDs to separate power (USB power might be simplest), so you can power cycle Teensy without also power cycling the LEDs.

Hi Paul,

I followed your suggestion and, unbelievably enough, I found that the problem is in the level shifter. :-O
I can't really understand how can a 74HCT245 enter this state... I hope it's a faulty device, otherwise I don't know what to think.

And why this happens almost always after uploading?

In case you have time to look at my (very simple) circuit, it's here:
https://github.com/lpaolini/Striptease/tree/main/hardware/EAGLE

Thanks a lot for pointing me to the right direction.

 

[KurtE]

Take this with a grain of salt, as I never use Eagle and the version I have is really old (6.6.0) and may not have read everything properly

But when I looked at your T4.1 board, it almost looks like some of the ground signals are not routed? C3, C2 and your 74245N?

Again probably not seeing it right...

 

[lpaolini]

Take this with a grain of salt, as I never use Eagle and the version I have is really old (6.6.0) and may not have read everything properly

But when I looked at your T4.1 board, it almost looks like some of the ground signals are not routed? C3, C2 and your 74245N?

Again probably not seeing it right...

OUCH! YOU'RE RIGHT!!

That's the problem, for sure!
At least it'll be easy to fix, because of the ground planes all around...

Thanks again!

 

[DL3HJG]

Hello,

I,m using T4.1 as a webserver for remote switch with 2 channels.
Application no critical
The data is transferred using the GET method.
Login with password
Table for status and switch
Logout or timeout
Only 1 client
Local button for switching
Mainloop toggle a LED every sec = 'Heartbeat'

Programming in Arduino-IDE V1.1.8.19 & Teensyduino V1.56

Librarys:
- QNEthernet
- Bounce
- elapsedMillis

For debugging I'm using 'Serial.print' about 10 times in the sketch for variables, html-request, pins etc.
Sometimes switches the ouput-pins to LOW after hours without user action.
I started a longtime log to Serial Monitor with timestamps on Serial Port COM8, no Teensy Port.
After 100 or 200 lines in 6hours 'Serial Monitor offline', significantly less than 64 k Bytes
I started again. The same.
Teensy Serial Monitor also.
With other serial software ('Cool Monitor) few minutes or one hour.
PC with Win7 Prof
- Energy Saving 'Always on' USB: 'No ernergy saving'

But the program on Teensy is running!
+ LED blinking
+ Webaccess possible
+ Local button can toggle the output
- Output is LOW without user action

? www attack ?​

May to much output on Serial? Out of memory?
Now i reduced the number of 'Serial.print'
No error in one day!

? Should I use a alternativ serial port: 1, 2 ?
? RasPi for serial logging? Software?

If it is of interest i can send the code.

Hans