Curt Welch
Member
We just had to hunt this bug down for work, and I thought I would share what I found.
We have a Teensy 4.1 as a controller in our robot that recently started rebooting for no known reason.
It would not just reboot once, but keep repeating every 10 seconds or so but when we shut it down and come back later it would be working fine again.
The solution is to remove the F1 fuse and just jumper it's location.
We suspected a heat related issue and it turns out that was the problem, but the part overheating was not any of the electronics,. It was the F1 PTC resettable fuse.
We are not using the Teensy 3.3V power for anything other than the parts on the Teensy 4.1 board. And we are not overloading the Teensy. And though we are using nearly every pin on the Teensy, we are not overdrawing current on any of them (as far as we know). All outputs at 4 mA or below.
But we are using heavy Ethernet IO, heavy CAN bus talking to 4 motors we are controlling in real time, 2 serial ports driving two other prop motors, I2C buss with heavy activity, and when we saw this start to happen, we had heavy USB I/O for debugging. I don't have an exact value for how much current the teensy is pulling at 5V, but it's probably less than 250 mA total. Probably below 200 mA. The 3.3V regulator on the Teensy is a 1 amp regulator so we are nowhere near the power limit of the regulator.
But all this CPU activity generates heat. The Ethernet chip generates heat, the regulator produces heat, and the PCT Fuse is also a heat-producing device. All this dumps heat into the PC board, which alters the trip curve of the PTC Fuse, that then causes it to trip at lower levels.
All our electronics are in an enclosed waterproof robot, so heat builds up on the inside. But at the time we saw this, the robot was open, and in our air-conditioned office, and was not being overly stressed with heat.
The CPU temperature for the Teensy was reporting around 70 °C when we first noticed the reboot problem.
What I know now is that the CPU will shut down due to heat, but only when it reaches 90 °C. And it will produce a CrashReport string explaining the cause of the shutdown, which is overheating. This boot cycle, when caused by F1, produces no CrashReport information. That's because it's simply losing power, turning off, and then back on, triggering the reboots.
We put a heat sink on the CPU and that lowered the CPU temp, but did not stop the problem. We modified the software to remove extra IO that was not needed and slow down control loops and messages. That helped, but didn't stop it.
The F1 fuse is a great safety feature for breadboarding because it protects the regulator from being blown out by shorting the 3.3V to ground or just by trying to power too many devices from the Teensy 3.3. However, in our industrial application, it caused the board to start failing and rebooting long before any of the chips reached critical levels of heat.
The PTC fuses also aren't designed to operate for long periods near their trip limits, nor do they hold up well when they are cycled too many times. So when we pushed the board to its limits and needed ot operate under heavy load in warm environments, the PTC was the first device to fail. Good for safety and protection in an office, not so good to have our industrial robot crash randomly and unexpectedly at such low temps.
I have seen numerous posts discussing heat-related crashes caused by overclocking or drawing too much current, but none I found mentioned that the F1 fuse was the actual cause of the crash, rather than hot electronics. I suspect most heat-related crashes are caused by this fuse being the first device to fail.
You can easily reproduce this problem by carefully warming up the Teensy 4.1 board with a heat gun. If you write code to print the CPU temp, you will see it starts crashing due to the F1 fuse when the CPU is only in the 60 °C range, well below what the CPU can withstand, which is more like 80 °C+. At least that is what I have seen. The parts are all cold to the touch on the outside when the CPU starts reporting 60 °C as its internal die temperature.
If you measure the 5V line after the fuse, you see the voltage starts to drop as the fuse gets warmer, and when it drops to below 3.3, the system crashes and reboots. As the fuse cools down, the voltage rises slowly back towards 5V. The overheat, cool-down cycle is what triggers the reboot loops.
So the PTC fuse protects all the electronics as intended, but limits how hard you can push the Teensy 4.1. The heat from the electronics alters the trip characteristics of the fuse, causing it to fail even when the current or heat limits of the electronics are still acceptable.
I wanted to share this in the Forums so that others who encounter the problem can understand it and know that removing the fuse or replacing it with a real fuse are options to consider for getting more heat resistance from the Teensy 4.1.
We have not yet conducted long-term testing with the fuse removed, so I am unsure if it causes any side effects when we run all the electronics at higher heat levels. We will find out...
Also to note, the fuse also powers and limits the current of the 5v out of the USB host port. We do not use that port so it's not an issue for me, but if you connect downstream USB devices and power it from the Teensy 4.1 board, this fuse limits how much current they can draw. Just be aware.
Curt Welch
curt@kcwc.com
Applied Impact Robotics, Inc.
We have a Teensy 4.1 as a controller in our robot that recently started rebooting for no known reason.
It would not just reboot once, but keep repeating every 10 seconds or so but when we shut it down and come back later it would be working fine again.
The solution is to remove the F1 fuse and just jumper it's location.
We suspected a heat related issue and it turns out that was the problem, but the part overheating was not any of the electronics,. It was the F1 PTC resettable fuse.
We are not using the Teensy 3.3V power for anything other than the parts on the Teensy 4.1 board. And we are not overloading the Teensy. And though we are using nearly every pin on the Teensy, we are not overdrawing current on any of them (as far as we know). All outputs at 4 mA or below.
But we are using heavy Ethernet IO, heavy CAN bus talking to 4 motors we are controlling in real time, 2 serial ports driving two other prop motors, I2C buss with heavy activity, and when we saw this start to happen, we had heavy USB I/O for debugging. I don't have an exact value for how much current the teensy is pulling at 5V, but it's probably less than 250 mA total. Probably below 200 mA. The 3.3V regulator on the Teensy is a 1 amp regulator so we are nowhere near the power limit of the regulator.
But all this CPU activity generates heat. The Ethernet chip generates heat, the regulator produces heat, and the PCT Fuse is also a heat-producing device. All this dumps heat into the PC board, which alters the trip curve of the PTC Fuse, that then causes it to trip at lower levels.
All our electronics are in an enclosed waterproof robot, so heat builds up on the inside. But at the time we saw this, the robot was open, and in our air-conditioned office, and was not being overly stressed with heat.
The CPU temperature for the Teensy was reporting around 70 °C when we first noticed the reboot problem.
What I know now is that the CPU will shut down due to heat, but only when it reaches 90 °C. And it will produce a CrashReport string explaining the cause of the shutdown, which is overheating. This boot cycle, when caused by F1, produces no CrashReport information. That's because it's simply losing power, turning off, and then back on, triggering the reboots.
We put a heat sink on the CPU and that lowered the CPU temp, but did not stop the problem. We modified the software to remove extra IO that was not needed and slow down control loops and messages. That helped, but didn't stop it.
The F1 fuse is a great safety feature for breadboarding because it protects the regulator from being blown out by shorting the 3.3V to ground or just by trying to power too many devices from the Teensy 3.3. However, in our industrial application, it caused the board to start failing and rebooting long before any of the chips reached critical levels of heat.
The PTC fuses also aren't designed to operate for long periods near their trip limits, nor do they hold up well when they are cycled too many times. So when we pushed the board to its limits and needed ot operate under heavy load in warm environments, the PTC was the first device to fail. Good for safety and protection in an office, not so good to have our industrial robot crash randomly and unexpectedly at such low temps.
I have seen numerous posts discussing heat-related crashes caused by overclocking or drawing too much current, but none I found mentioned that the F1 fuse was the actual cause of the crash, rather than hot electronics. I suspect most heat-related crashes are caused by this fuse being the first device to fail.
You can easily reproduce this problem by carefully warming up the Teensy 4.1 board with a heat gun. If you write code to print the CPU temp, you will see it starts crashing due to the F1 fuse when the CPU is only in the 60 °C range, well below what the CPU can withstand, which is more like 80 °C+. At least that is what I have seen. The parts are all cold to the touch on the outside when the CPU starts reporting 60 °C as its internal die temperature.
If you measure the 5V line after the fuse, you see the voltage starts to drop as the fuse gets warmer, and when it drops to below 3.3, the system crashes and reboots. As the fuse cools down, the voltage rises slowly back towards 5V. The overheat, cool-down cycle is what triggers the reboot loops.
So the PTC fuse protects all the electronics as intended, but limits how hard you can push the Teensy 4.1. The heat from the electronics alters the trip characteristics of the fuse, causing it to fail even when the current or heat limits of the electronics are still acceptable.
I wanted to share this in the Forums so that others who encounter the problem can understand it and know that removing the fuse or replacing it with a real fuse are options to consider for getting more heat resistance from the Teensy 4.1.
We have not yet conducted long-term testing with the fuse removed, so I am unsure if it causes any side effects when we run all the electronics at higher heat levels. We will find out...
Also to note, the fuse also powers and limits the current of the 5v out of the USB host port. We do not use that port so it's not an issue for me, but if you connect downstream USB devices and power it from the Teensy 4.1 board, this fuse limits how much current they can draw. Just be aware.
Curt Welch
curt@kcwc.com
Applied Impact Robotics, Inc.