F1 PTC Fuse can cause heat-related reboot loops on heavily loaded Teensy 4.1 projects.

[Curt Welch]

We just had to hunt this bug down for work, and I thought I would share what I found.

We have a Teensy 4.1 as a controller in our robot that recently started rebooting for no known reason.

It would not just reboot once, but keep repeating every 10 seconds or so but when we shut it down and come back later it would be working fine again.

The solution is to remove the F1 fuse and just jumper it's location.

We suspected a heat related issue and it turns out that was the problem, but the part overheating was not any of the electronics,. It was the F1 PTC resettable fuse.

We are not using the Teensy 3.3V power for anything other than the parts on the Teensy 4.1 board. And we are not overloading the Teensy. And though we are using nearly every pin on the Teensy, we are not overdrawing current on any of them (as far as we know). All outputs at 4 mA or below.

But we are using heavy Ethernet IO, heavy CAN bus talking to 4 motors we are controlling in real time, 2 serial ports driving two other prop motors, I2C buss with heavy activity, and when we saw this start to happen, we had heavy USB I/O for debugging. I don't have an exact value for how much current the teensy is pulling at 5V, but it's probably less than 250 mA total. Probably below 200 mA. The 3.3V regulator on the Teensy is a 1 amp regulator so we are nowhere near the power limit of the regulator.

But all this CPU activity generates heat. The Ethernet chip generates heat, the regulator produces heat, and the PCT Fuse is also a heat-producing device. All this dumps heat into the PC board, which alters the trip curve of the PTC Fuse, that then causes it to trip at lower levels.

All our electronics are in an enclosed waterproof robot, so heat builds up on the inside. But at the time we saw this, the robot was open, and in our air-conditioned office, and was not being overly stressed with heat.

The CPU temperature for the Teensy was reporting around 70 °C when we first noticed the reboot problem.

What I know now is that the CPU will shut down due to heat, but only when it reaches 90 °C. And it will produce a CrashReport string explaining the cause of the shutdown, which is overheating. This boot cycle, when caused by F1, produces no CrashReport information. That's because it's simply losing power, turning off, and then back on, triggering the reboots.

We put a heat sink on the CPU and that lowered the CPU temp, but did not stop the problem. We modified the software to remove extra IO that was not needed and slow down control loops and messages. That helped, but didn't stop it.

The F1 fuse is a great safety feature for breadboarding because it protects the regulator from being blown out by shorting the 3.3V to ground or just by trying to power too many devices from the Teensy 3.3. However, in our industrial application, it caused the board to start failing and rebooting long before any of the chips reached critical levels of heat.

The PTC fuses also aren't designed to operate for long periods near their trip limits, nor do they hold up well when they are cycled too many times. So when we pushed the board to its limits and needed ot operate under heavy load in warm environments, the PTC was the first device to fail. Good for safety and protection in an office, not so good to have our industrial robot crash randomly and unexpectedly at such low temps.

I have seen numerous posts discussing heat-related crashes caused by overclocking or drawing too much current, but none I found mentioned that the F1 fuse was the actual cause of the crash, rather than hot electronics. I suspect most heat-related crashes are caused by this fuse being the first device to fail.

You can easily reproduce this problem by carefully warming up the Teensy 4.1 board with a heat gun. If you write code to print the CPU temp, you will see it starts crashing due to the F1 fuse when the CPU is only in the 60 °C range, well below what the CPU can withstand, which is more like 80 °C+. At least that is what I have seen. The parts are all cold to the touch on the outside when the CPU starts reporting 60 °C as its internal die temperature.

If you measure the 5V line after the fuse, you see the voltage starts to drop as the fuse gets warmer, and when it drops to below 3.3, the system crashes and reboots. As the fuse cools down, the voltage rises slowly back towards 5V. The overheat, cool-down cycle is what triggers the reboot loops.

So the PTC fuse protects all the electronics as intended, but limits how hard you can push the Teensy 4.1. The heat from the electronics alters the trip characteristics of the fuse, causing it to fail even when the current or heat limits of the electronics are still acceptable.

I wanted to share this in the Forums so that others who encounter the problem can understand it and know that removing the fuse or replacing it with a real fuse are options to consider for getting more heat resistance from the Teensy 4.1.

We have not yet conducted long-term testing with the fuse removed, so I am unsure if it causes any side effects when we run all the electronics at higher heat levels. We will find out...

Also to note, the fuse also powers and limits the current of the 5v out of the USB host port. We do not use that port so it's not an issue for me, but if you connect downstream USB devices and power it from the Teensy 4.1 board, this fuse limits how much current they can draw. Just be aware.

Curt Welch
curt@kcwc.com
Applied Impact Robotics, Inc.

 

[BriComp]

Thanks for reporting the above. It's interesting and very useful information.

 

[Curt Welch]

I did a bit more testing yesterday on a Teensy 4.1 board with the PTC fuse removed. By overclocking the CPU I could force it to draw more current and produce more heat. On a board with the fuse removed, I could overclock all the way to the max of 996 MHz without crashing. And with no heat sink on the chip. But then later (after some small code changes), the same board would not work at 996 Mhz (random hangs) but would still work up to 972 MHz. My test code was also doing full speed CAN bus IO to a motor to generate more heat.

With no PTC fuse, and running at 972 MHz and maximum CAN IO, on a Teensy 4.1 with no Ethernet chip, it was drawing 230 mA from the USB that powered it, and the internal temperature of the CPU reached 80 °C. And it was stable.

Grabbing a different Teensy 4.1, with the normal PTC fuse on it and an Ethernet Chip, I did some similar testing. That would work with the same stress software doing CAN IO, up to an overclocking of 804 MHz. Internal CPU temp was only 60 C and it was pulling 189 mA from the USB.

But at 852 Mhz this second Teensy with PTC draws 201 ma, and almost immediately goes into the PTC reboot loop. Internal CPU temp was only 61C.

It seems the PTC on this board starts to trip far lower than I suspected, since it's around 201 ma total for the board. And at that low of a CPU temp I would not expect the CPU to be heating up the board all that much. So that makes me wonder what version of PTC chips were installed on this board since it seems to be tripping at only 200 mA.

This testing was all just done at my desk in an air-conditioned office using off-the-shelf Teensy 4.1 mounted in breadboards.

So just pulling as little as around 200 mA total for the Teensy 5V supply seemed to set off the PTC fuse on this one board.

I don't know where we bought these Teensy 4.1 boards. Are there multiple manufacturers selling Teensy 4.1 boards?

I have to wonder if they might be putting too small of a PTC fuse on the boards? The schematic refers to it as a 500 mA fuse, without specifying any part numbers. Is there an official design for the board that specifies recommended part numbers for this fuse?

I could not read any markings on the fuses.

 

[KenHahn]

PJRC manufactured the Teensy up until March '25 and then turned the manufacturing rights over to SparkFun which has manufactured them since. To my knowledge, there were no intentional functional changes when the manufacturing changed hands. Looking at the PTC on both manufacturing versions, they appear to be identical.

I just hooked one of the latest SparkFun built Teensy 4.1 up on a breadboard and connected a load to the 3.3V regulator and set it for 300mA since there were no other loads on any of the pins and the Teensy is just running blink. The setup is drawing 360mA from USB. The input voltage to the PTC is 5.07V and drops to 4.84V on the output side and I let it run for about 15 minutes with no issues noted.

Went up to 400mA on the output of the 3.3V regulator. USB draw is now 450mA. Voltage on output side of the PTC is now 4.35V. Setup has been running for about another 30 minutes with no issues. PTC temp measures 39C in 21C ambient. This should be getting near the max rating of the PTC, so I am not seeing any issue with the PTC current handling at least with the Teensy 4.1 that I am testing with.

 

[Curt Welch]

This morning, I pulled a brand-new Teensy 4.1 out of the box and have been testing it. So far, I've seen no PTC fuse issues on the new board. I've only run at 200 mA loads so far but was getting ready to add some external load to the 5V circuit to find where a new PTC fuse will trip. I didn't expect to see anything different than what you saw, Ken, at this point.

I suspect the issue is that the PTC fuses on our other boards have degraded by being overloaded or overheated to the point that they are now triggering below their standard rated temperatures. I don't know the full history of either of the two old boards I've been testing so I can't say for sure what they were subjected to. We use lots of Teensys for various projects around here and reuse them as needed.

The one in our current production robot that started all this has seen higher temps in its operating environment, but probably not much more than 40C (100F)?. It was out in the Texas sun for extended periods. But it could have gone higher. We don't yet have a working ambient temperature probe inside the robot to tell us.

I must assume at this point that the Teensy design is good, as long as you don't overload it for too long or run it in an excessively hot environment while under heavy loads for too long.

The moral of this story seems to be that if you see a Teensy 4.1 in a restart loop every few seconds, with no other obvious causes, such as bad code or power problems, check to see if the PTC fuse is cycling. And if you want a Teensy to have more resilience in a hot environment, consider removing the PTC fuse or replacing it with a less heat-sensitive fuse.

At this point, I've seen this restart bug on three of our Teeny's, so it's a problem that we have managed to create multiple times. And no one here remembers buying them from any other place than PJRC or one of your resellers, so I don't think this is a third-party knock-off problem (if those even exist). So we probably just abused them into this condition!

 

[KenHahn]

I am almost positive that there are no clones of Teeny 4.1 floating around.

I have read that PTC fuses can weaken over time if subjected to repeated tripping/resetting cycles or perhaps just fairly severe thermal cycles. That doesn't seem like it would be a factor in your case, but maybe.

 

[Curt Welch]

With the new board, I've now added 320 mA of external load on the 3.3, with the board drawing 162 ma with no external load, overclocked at 804 Mhz, and no problems wth the PTC.

Total USB load is 487 mA. PTC output is 4.6V so it's dropped some but nowhere near what it takes to crash the processor.

The internal temperature of the processor is up to 71 °C. It was 60 °C with no external loads. So even though the resistor loads are external in the breadboard, the extra current is heating up the regulator and PTC, and that extra heat drives the CPU temp up +10C.

So yeah, my testing duplicated what you saw, Ken. With a new board, the PTC hasn't been an issue even with 150 mA of load on the board and 320 mA of external load (above the 250mA recommendation).

So either the PTCs on my other boards degraded, or there was a batch of Teensy's produced with lower trip point PTCs in the past. I lean towards believing we abused the boards and degraded the PTCs. I'm going to leave this heavily loaded Teensy 4.1 running over the weekend and see if it holds up ok and see if there is any evidence of that changing the PTC behavior.

 

[sicco]

Curt, can you share a picture on how your T4.1 is fitted in the target system? One that rules out that other electronics heats up the environment where the Teensy and its PTC sits.
Plus are you using SD cards? Some cards use very high currents occasionally - i think when writing to them.

 

[Curt Welch]

Curt, can you share a picture on how your T4.1 is fitted in the target system? One that rules out that other electronics heats up the environment where the Teensy and its PTC sits.
Plus are you using SD cards? Some cards use very high currents occasionally - i think when writing to them.

Sure. First, no, we are not using the SD at all. No card in the slot.

The robot where we run the teensy is FULL of heat-producing electronics and motors. The teensy itself is mounted on a PC board that is stuffed with other electronics and has an Ethernet switch board right next to the Teensy, a very hot Ethernet to fiber media converter a few inches away, and 10mm below the board the teensy is mounted on is another board of the same size that has all the power handling swtiches for turning motors and other devices on and off (controlled by the Teensy). There is an electronics box below the board that processes UT data, and above these boards is a single-card Windows desktop-class computer. About 10mm to the left of the teensy is the compartment that holds the power supplies that convert 400V DC to 48V, 12V, and 5V, and it produces a good amount of heat, but that heat is conducted chiefly into the shell of the robot or out through the external heat sinks. 6" or so behind the electronics and Teensy are a pair of 300W pulsation motors. They are isolated in a separate chamber, but when running, they produce a good bit of heat that is also conducted through the aluminum body of the robot.

There's heat everywhere. Trying to cope with the heat in this robot has always been a never-ending challenge for us.

This robot operates underwater. And when underwater, all this heat is conducted into the water. But the robot spends time out of the water as we prep it or work on it. And it has spent many minutes sitting in the hot sun while powered up.

However, until last week, we had never seen this reboot problem in all our operations. (This machine is still very much in R&D development.) We might have seen it a year ago, but we didn't understand what it was at that time. We are on a new version of the robot now. When we first saw it start to reboot, it was in the office, and the fear motors were taken off and the Windows computer above the teensy was removed (we are replacing it). The normally closed cavity was open and free for air to circulate a bit. And the overall power usage was low compared to normal operations. So it was under cooler operating conditions than normal when we started to see this.

What was different is that out software guy was testing some new arm motion control software that put a heavy CPU on the Teensy, because the prime logic fo this is running on an external server and sending a high-speed stream of motor commands to the Teensy. so lots of Ethernet IO, and lots of CAN IO. At the same time he had added a lot of serial IO debug to the Teensy going out the USB port. That adds more heat and CPU load to the Teensy (per my testing).

Yes, it's operating in a hot environment, and it has a heavy CPU load at the time. But that heat level when the robot is open never rises to more than "some parts of the robot are a little warm to the touch (aka 30C on the outside) that we have seen.


Here's the robot with the end up as it was when we first saw this problem. The Teensy is behind that mass of wires.

I don't have a good picture on my phone of the electronics boards, but here's an older one when it was half torn apart on the bench being worked on many months ago:

The teensy plugs into the lower left corner of the board, and you can see how many other electronics are near it.

However, when I tested this Teensy (in the picture below), on my desk with nothing attached to it:

(Which I believe was once in our last version of this robot and subjected to high operating heat)

It starts to trigger the PTC and go into a reboot loop if I run it at 852 MHz. The CPU temp is only 53. I'm in an air-conditioned office.

This debug output shows it ran for 14 seconds before it started to crash. It rebooted and ran for 3 seconds, then rebooted constantly.

This Teensy is drawing 170 mA max on the USB, and goes up and down as it reboots.

The +5V side of PTC shows 5.1 or 5.2 V. The other side of PTC jumps from 4.8V or so down to the 3.3V range and back and forth as it go though this reboot cycle. Hard to see well using only a meter but that's approximately what's happening.

So the PTC on this board is not working as it does on a new board, has no problem at all with this same test.

So the PTC on this board is not working as a new one does, that I was testing yesterday, and as Ken saw in his testing.

But this Teensy was likely in a hot robot for extended periods and degraded the PTC. So now the thing is cutting power at only 170 mA.

So the result of my understanding is if you want a Teensy to operate in higher loads in warm to hot environments, it's this PTC that will start to fail first, long before the electronics fail. Therefore, removing or replacing the PTC will provide higher heat resistance.

 

[KenHahn]

That is some serious looking hardware! I was dubious that you could be stressing the PTC that much as I have never heard of this PTC issue with Teensy before, but it looks like you are indeed sending Teensy where no Teensy has probably gone before.

I spent some time trying to find any info on long term heat degradation of PTC, but couldn't find anything useful other than just the vague statements that high heat thermal cycles can cause it to degrade over time.

For reference, this is the actual PTC used on the board per a post by Paul: https://www.digikey.com/en/products/detail/bourns-inc/MF-FSMF050X-2/2039256

Possible degradation aside, operating the PTC at a high temperature will lower the trip current since the PTC is basically just a thermistor that self-heats and then knees over. Here is the thermal derating table from the spec sheet. Rated at 500mA @ 23C, but derates quite a bit as ambient temps go up from there.

In your application where the power draws are well understood and fixed and a fairly hot environment, I think bypassing the PTC as you are doing is probably the best course of action and I can't imagine any long-term consequences. You could always put a mechanical fuse in the 5V line if you wanted to implement some type of current protection on that circuit.

 

[Curt Welch]

Thanks for that reference to the actual PTC Ken. I had found something on the internet that pointed to a different brand but otherwise same basic specs. 500 mA hold, and 1A trip point. That sounded like a good rational choice since the regulator can handle 1A, and the normal loads are all under 500 mA.

I left a new Teensy running all weekend, pulling 500 mA with the help of external resistors on the 3.3V output of the board. I have it overclocked at 856 MHz and running high-speed CAN IO to a motor, which pushes the board current alone (without the external resistors) up to around 150 mA.

The PTC has not tripped on this Teensy with these loads.

And this morning I find it running fine still, but it is curiously running much hotter per the CPU temp reading. On Friday, it stabilized around 72C. Now it' up to 77C so far after 28 minutes since I plugged it into my laptop. So, something in the system seems to be changing due to the 3-day exposure to higher operating temperatures. I don't know what changed, however.

It seems to me that the most likely cause of PTC degrading is not just heat, but forcing it through its trip cycle. I don't understand the physical structure of changes that happen, but if it's breaking bonds in a carbon layer due to the substrate it's connected to expanding, then it seems quite likely that repeated breaking and and-reforming those bonds could well lead to a crack growing larger which then means it opens with less strecheing lower temps).

I'm tempted to set up a current-controlled test that forces it to keep cycling through the trip point and then cooling down, and see how that trip point evolves and how fast it degrades. However, I've already spent too many work hours on this, so the test might not happen.

Yes, for us, the PTC doesn't make sense, so removing it is the right course of action. We have well-defined current limits in our design, but we lack reasonable control over environmental heat; so we need it to keep working until the silicon goes into thermal runaway, and we will then catch that with software heat shutdown limits later.

 

[MarkT]

It seems to me that the most likely cause of PTC degrading is not just heat, but forcing it through its trip cycle.

But the trip cycle _is_ heat. Polymers have non-trivial amounts of creep, which could explain deterioration after (repeated or long-term) tripping. The particles are embedded in the polymer, expansion of the polymer pulls the particles apart, contraction forces them together. The polymer also has a gradual phase change from crystalline to amorphous with increasing temperature which increases the thermal expansion I suspect.

 

[Curt Welch]

But the trip cycle _is_ heat. Polymers have non-trivial amounts of creep, which could explain deterioration after (repeated or long-term) tripping. The particles are embedded in the polymer, expansion of the polymer pulls the particles apart, contraction forces them together. The polymer also has a gradual phase change from crystalline to amorphous with increasing temperature which increases the thermal expansion I suspect.

Yeah, I'm aware that tripping is heat. That's why I wrote "not just heat". The other part I was looking at on Digikey specified its trip temperature as over 200C. So not just hot, but very hot!

But my thought is that the trip point will degrade faster, not if you just hold it at a high temp near the trip point (say the temp generated by 900 mA). But if you actually are passing through the transition, that makes it trip. Maybe the heat is all that is needed, that was my question to wonder about!

But reading the spec sheet linked above, I just noticed something that the other spec sheet I had studied did not include:

Trip Cycle Life - Vmax, Imax, 100 cycles

Vmax is 6V, and Imax is 40A for the part used on the Teensy 4.1.

This gives a general idea that the part is not expected to trip a lot and still meet its specifications. But the question remains, how fast does it decline in practice? Our system was resetting in a boot loop at around 10 seconds per cycle. 16 minutes of being stuck in that boot loop would exceed the 100-cycle limit of the PTC. So if it's resetting and not being noticed, it could cycle 1000s of times.

I don't know what we did to ours (what heat levels or reboot cycles), but we messed them up.