Brainstorming possible causes of Ethernet link loss

S

shawn

Well-known member
I’m looking for some thoughts because I’m stuck on how to even debug this. I can’t reproduce it, but a customer sees this from time to time.

So I have a project where everything runs fine over a network. It’s a lighting controller that receives lots of sACN UDP packets. Once in a while, the Ethernet activity just stops. That is, until the ribbon cable attached to the Teensy 4.1 is unplugged and re-plugged. Unplugging and re-plugging the Ethernet cable doesn’t seem to help, just unplugging and re-plugging the ribbon cable.

Based on some minimal debugging, I’ve confirmed that the system thinks that there’s no link. When the ribbon cable is reseated, the system regains the link. The strange part is reseating the Ethernet cable does not restore the link.

I’m trying to brainstorm possible reasons, and I would love others’ thoughts. Here’s what I have so far:
1. EMF
2. The switch/router drops the link (I’ve asked for logs, but I haven’t gotten them yet)
3. My Teensy 4.1 driver (inside QNEthernet)
 
Last edited:
@PaulStoffregen might know what the ribbon cable re-plug might do to the PHY chip?

Isn't there an enable pin to the chip? Can that be toggled off on to maybe trigger the same behavior/reset?

Is this happening on more than one device?
 
It’s happening on many devices. I’m going to check my link-related driver code and investigate the schematics.

Is it possible that EMF can be a cause of something like this, say with power supplies that are known to have bad radiating noise? With the particular power supply I'm thinking of (I don't have the model number just now Lumipro P-200S-24-ETL), just bringing it near 3-wire LEDs makes them fail and appear to cause data to be skipped to the next pixel.
 
Last edited:
Looking at this circuit, the Teensy Ethernet kit connects the Teensy ground to the Ethernet cable. That seems to me to be vulnerable to EMC … not familiar enough with good Ethernet practice to know if that’s standard or not. I always thought it was isolated by the magnetics up to however many kV they were specced to…
 
I'll add some more info about the hardware: Teensy 4.1, Ethernet Kit (PJRC version), double-high pins into an OctoWS2811, likely unshielded Ethernet cable for both the LED outputs and the network connection.
 
If I were to build up a couple alternate adapters with different capacitors, would you be able to give them a try in these problematic applications?
 
The magjacks used with the Ethernet module have a 1000pf capacitor as well as a 75 ohm resistor in series between the cable ground and the Teensy ground. This includes both the Cetus J1B1211 and the Link-PP JPJ4012AHNL. Paul's example schematic doesn't show that cap for some reason.

Here is the schematic from the Cetus spec sheet.
1769209611166.png

EMF/EMI could definitely cause the link to drop according to AI and your LED test seems to clearly show your power supply is radiating pretty badly and a likely culprit in my opinion.

Reseating the cable will cause the link to be renegotiated. There is a 0.1uF cap between the center taps on pins 2 & 5 to Teensy ground on the Ethernet module as well as the magjack magnetics which would be disconnected from the Teensy circuit when the jumper cable is removed and would stay in circuit when the Ethernet cable is removed. I have no clue why that would make a difference from an electrical standpoint with the PHY chip.

If you do use a shielded cable just be aware that the PJRC module does not ground the shield, so it would need to be grounded on the other end. A shielded cable that isn't grounded at either end can cause other issues.
 
@KenHahn - Thanks for these notes. Maybe my driver needs to be more aware of a need for link renegotiation. Maybe this can, at least partially, be mitigated in software.

@PaulStoffregen - If you have the parts, may I request 3 or 4 of them? (Or whatever you can supply, if not. I'll put them in more than one place.)
 
@KenHahn Do you know the reason why reseating the Ethernet cable won't cause auto-negotiation but reseating the ribbon cable will? I'm going through the DP83825i docs but nothing's jumping out at me.
 
I just ordered fresh PCBs from OSH Park, both the PJRC version and something very similar to SparkFun's version. I got only 3 pieces of each.
 
Update: I just added a driver_restart_auto_negotiation() function to the drivers (the Teensy 4.1 driver is the only one that does something), and I'm now calling this on link-down in the project (not in the driver). This is in the hopes that I actually get a link-down notification. Note that I haven't tried it out yet with the project. More updates forthcoming.

The gory details: That function sets bit 9 of the BMCR register to '1'. (Page 44 of the spec.)

The QNEthernet library branch is: restart-auto-neg.
 
Last edited:
Update: after implementing a call to driver_restart_auto_negotiation() when link-down is detected, the Teensy managed to stay on the network. So far. It’s a positive sign, but a few more days will tell us more.
 
Update: It's been a few days, and I haven't seen the Ethernet freeze since implementing the renegotiate-on-link-down fix. It seems to be working.

Here's some sample code (with a bonus abortAll() call):
C++: 
#include <QNEthernet.h>

using namespace qindesign::network;

// Forward declare this
extern "C" void driver_restart_auto_negotiation();

void setup() {
  // ...some code...

  Ethernet.onLinkState([](bool state) {
    if (state) {
      printf("[Ethernet] Link ON\r\n");
    } else {
      printf("[Ethernet] Link OFF\r\n");

      // I also do this because Windows systems forget TCP connections
      // This causes the connections to take forever to close
      internal::ConnectionManager::instance().abortAll()

      // Help mitigate link drop when EMF
      driver_restart_auto_negotiation();
    }
  });
 
  // ...more code here...
 }
 
I just realized that all the driver functions are included in <QNEthernet.h>. There’s no need to do that forward declaration.
 
I've also got this problem now, and the solution mentioned above does not work for me. I can't get the link back up without reseating the ribbon cable to the network jack, or power cycling the Teensy.
I can never remember having this problem before now, but I've also got it to happen with the NativeEthernet library, so it's certainly not a QNEthernet regression.
 
You must log in or register to reply here.
Back
Top