Understanding how I killed Teensy 4.0 Serial1 with simple circuit

V

VictorFS

Well-known member
I have an application where one Teensy 3.5 (main module) communicates with a Teensy 4.0 (remote module) over a 10 meter wire.

This wire carries the TX and RX signals from Teensy 3.5 Serial6 to Teensy 4.0 Serial1, GND and 12V that powers the remote. The 12V line goes to a step-down to convert to 5V that powers the remote Teensy. In the middle of the wire, there is a male and female 4-pin connector plug which allows for quick connections and disconnections of the modules.

20240411_104428.jpg


Everything was working fine, and the communication between the Teensies was working flawlessly. While performing some testing on the part of the code that detects disconnections between the two modules and tries to reestablish the communication, I had to simulate a physical disconnection between the modules, and did it simply disconnecting the before-mentioned plug a few times (with both teensies powered). I did this a couple of times before the Teensy 4.0 Serial1 suddenly stopped communicating. While troubleshooting the issue, I detected that Serial1 died and that the multimeter detects a short between pin 0 (RX) and GND.

I then replaced the Teensy 4.0 for a new one and repeated the procedures. The exact same thing happened: after a couple of connections and disconnections of the plug, Serial1 of Teensy 4.0 died with same symptoms (short circuit between RX and GND while powered off). I could not find any explanations for this so far, as the circuit works normally and for a long time while everything is connected, but kills the Teensy 4.0 serial after a few disconnects. All the rest of the circuit continues to work fine. Even the Teensy 4.0 continues to work, blink, print characters on the USB serial, etc after having the Serial1 damaged.

Do anyone have an idea of what could be happening? Is there any easy way to protect the Teensy 4.0 serial lines to prevent this from happening again?
 
Any chance the 4 pin connector could be plugged in backwards by mistake? Or even if it has a shape that can't fully mate backwards, is it possible for the 12V to touch the other pins even briefly if someone tries to plug it in the wrong way?
 
Hi Paul!
The connector has a specific orientation that prevents it from being plugged in any other way. Check the image below:
71LRNtmSQaL.jpg

Even so, after the first Teensy was damaged, I took extra care to plug and unplug the connector the right way and without jiggling. The new Teensy 4.0 still suffered the same issue :(

I was thinking maybe it has something to do with the long wire? I don't have much knowledge of electronics, but could it be some inductance effect of the wire or maybe ESD during the power off/on procedure of the remote module (containing the Teensy 4.0) ?

Note that when I unplug the cable, I also cut the GND connection and the 12V connection. When I reconnect, both are reestablish at the same time and the teensy boots. The main module with Teensy 3.5 remains powered all the time.
 
Does the connector have a design that ensures the ground pins mate first?
Generally when hot plugging you want ground to connect first then power and only then the signals.
If power and your data signals somehow connect before the ground it can look like a high impedance -12V source on the data pins (I've blown up PC USB ports in the past due to something similar).

Even if everything connects at the same time then it will take a short while for the voltage out of the regulator to come up to the correct level, during that time you'll in effect be trying to power the teensy through the serial lines.

Try adding a series resistor (100 ohms or so) on the data lines followed by some clamping diodes (i.e. a bat54-s between GND, signal and 3.3V)
 
AndyA said:
Does the connector have a design that ensures the ground pins mate first?
Click to expand...
No, it doesn't. Most of the times, all of them will connect the same time. Depending on the inclination of the plug or if any of the pins are loose, any of them could connect first than the other.
AndyA said:
If power and your data signals somehow connect before the ground it can look like a high impedance -12V source on the data pins
Click to expand...
That's a very interesting take. Haven't tough of this before .
AndyA said:
Try adding a series resistor (100 ohms or so) on the data lines followed by some clamping diodes (i.e. a bat54-s between GND, signal and 3.3V)
Click to expand...
Nice ideia! Would it be something like this?

1712846545129.png



Is it ok to use this with the high speed digital serial lines? I imagined that maybe resistor of the diodes could interfere with the rise/fall times.

If I implement this idea, do you think that I could protect the circuit even with the current connector design (which allows any pin to connect first) or I would still need to rethink the connector? Can I implement this circuit on the main module side (Teensy 3.5) or it needs to be near the remote module (Teensy 4.0) ?
 
VictorFS said:
Nice ideia! Would it be something like this?

Is it ok to use this with the high speed digital serial lines? I imagined that maybe resistor of the diodes could interfere with the rise/fall times.

If I implement this idea, do you think that I could protect the circuit even with the current connector design (which allows any pin to connect first) or I would still need to rethink the connector? Can I implement this circuit on the main module side (Teensy 3.5) or it needs to be near the remote module (Teensy 4.0) ?
Click to expand...
Yes, just like that. It should be on the teensy4 side between the part being protected and the connector. You are protecting against invalid voltages on the connector so it needs to be after the connector.
It's a serial port, it's not that high speed. 100 ohms should be fine but you could always make it lower if you want, the aim is to ensure that the worst case current through the diodes is something manageable. In this case you could get 12V so 100 ohms clamps it to 120mA.

I think with the protection you'll be ok with that connector.

To be honest putting TTL serial data over a 10 meter cable doesn't sound like the best plan to start with, there is a reason RS232 exists.
The ideal solution would be to throw in a couple of line drivers and run RS232 signal levels down that cable. Power the drivers off the Teensy 3.3V output and you'll get all the protection you need against connection and power sequencing.
 
Thanks for the support!

I will try to incorporate RS-232 on my next circuit design. Right now, it doesn't have the space or connections for the needed RS-232 transceiver, like the MAX3232. I will try to solve the issue with the diodes and the resistor in the current circuit board version.
 
PaulStoffregen said:
Sorry if I missed this detail earlier, can you say how long the wire is and any other details about the physical construction we can't see from the diagram?
Click to expand...
Me too. But it was in his first line, which my eyes missed on the first readings.
VictorFS said:
I have an application where one Teensy 3.5 (main module) communicates with a Teensy 4.0 (remote module) over a 10 meter wire.
Click to expand...
 
I would still worry about 12 volts on the same connector if this will ever be hot plugged. For example, imagine what might happen if the 12V pin and one or more of the data pins mate before GND.

Modern connectors meant for hot plugging usually have longer GND pins, so you can be certain GND is connected before anything else.
 
PaulStoffregen said:
Oh, yeah, 10 meters is definitely a job for RS232 or RS485.
Click to expand...
Nice! So far we have been using just normal 3.3V UART TTL over those 10m wires and it was working normally, while I agree that RS232 or RS485 would be more suitable for the job.

The cables are just regular multiconductor cable, unshielded and not twisted. We have been having success with this setup so far, communication wise. Only now that I've been testing the hot-plugging that I've faced problems.
 
PaulStoffregen said:
I would still worry about 12 volts on the same connector if this will ever be hot plugged. For example, imagine what might happen if the 12V pin and one or more of the data pins mate before GND.

Modern connectors meant for hot plugging usually have longer GND pins, so you can be certain GND is connected before anything else.
Click to expand...
Honestly, I have never tough about this problem before and am now concerned.

Considering a scenario were I can't change the plug for a newer one specifically designed for hot plugging, is there any easy solution to protect the circuit from this situation (12V pin and data pin connected before GND)?

Maybe I didn't face problems like this before out of pure luck, but could still be stressing the components.
 
VictorFS said:
I have an application where one Teensy 3.5 (main module) communicates with a Teensy 4.0 (remote module) over a 10 meter wire.

This wire carries the TX and RX signals from Teensy 3.5 Serial6 to Teensy 4.0 Serial1, GND and 12V that powers the remote. The 12V line goes to a step-down to convert to 5V that powers the remote Teensy. In the middle of the wire, there is a male and female 4-pin connector plug which allows for quick connections and disconnections of the modules.

View attachment 33994

Everything was working fine, and the communication between the Teensies was working flawlessly. While performing some testing on the part of the code that detects disconnections between the two modules and tries to reestablish the communication, I had to simulate a physical disconnection between the modules, and did it simply disconnecting the before-mentioned plug a few times (with both teensies powered). I did this a couple of times before the Teensy 4.0 Serial1 suddenly stopped communicating. While troubleshooting the issue, I detected that Serial1 died and that the multimeter detects a short between pin 0 (RX) and GND.

I then replaced the Teensy 4.0 for a new one and repeated the procedures. The exact same thing happened: after a couple of connections and disconnections of the plug, Serial1 of Teensy 4.0 died with same symptoms (short circuit between RX and GND while powered off). I could not find any explanations for this so far, as the circuit works normally and for a long time while everything is connected, but kills the Teensy 4.0 serial after a few disconnects. All the rest of the circuit continues to work fine. Even the Teensy 4.0 continues to work, blink, print characters on the USB serial, etc after having the Serial1 damaged.

Do anyone have an idea of what could be happening? Is there any easy way to protect the Teensy 4.0 serial lines to prevent this from happening again?
Click to expand...
Hot-plugging requires a connector where ground & power connects before data, as in a USB plug. Otherwise in your case you can get a situation where only the 12V and a data line are connected, putting 12V through the Teensy.

Alternatively add opto-isolators on the data connections so they are galvanically isolated.

If you add series resistors their value can be quite large if the serial data rate is low, which will protect better.

The classic resistor + BAT54S protection circuit is another candidate.
 
RS232 might mitigate most of the concerns. Look for RS232 chips which can handle accidental connection to 12V.

You might also consider a PTC fuse on the 12V supply side. They won't protect Teensy from damage, but if something goes wrong in the cable or connector wiring that causes 12V to short to GND, at least you'll limit the current in the cable. Most PTC fuses can allow about 2X their rated current sustained or about 10X for short times. So a 150 or 200 mA PTC fuse can keep wires from burning or insulation from melting if things go horribly wrong.
 
Another concern to consider with hot plugging is the inrush current during connect and high frequency effects (mainly due to self inductance of the wire) at connect and disconnect.

The USB 2.0 spec has an excellent summary of the issues in section 7.2.4 on page 177 (the 205th page of the PDF). Here's the PDF.


Look for the "flyback voltage" stuff near the bottom of that page.
 
That's great info. I'll check that out and see what I can do to improve my current circuit.
Thank you all for the help!
 
You must log in or register to reply here.
Back
Top