UDP Ethernet Hangs randomly at Udp.parsePacket()

airpanther

Active member
Good day,

I've been using the Ethernet.h library for a while, and generally it works great! I can send and receive UDP broadcast messages to/from other devices. Challenge is, incoming UDP messages simply stop working after a random interval. Sometimes it happens after 1-2 minutes, and sometimes longer. Other times, I can run the sketch for hours without a single hiccup. I simplified the code and added a bunch of traps to see where the sketch was hanging up, and it always hangs at the same line:

int packetSize = Udp.parsePacket();

Adding a delay(1) helped, and adding a delay(10) helped even more, but eventually it still stops working. I also have a keep alive, which sends a small UDP message every 2 seconds. It still continues to send successfully when the incoming messages stop. I've tried running the code in a thread with the same results, I've tried using a long instead of an int, no luck.

Any idea why the incoming UDP comms randomly stop working, while the outgoing UDP continues working fine? I'm using a Teensy 3.5 and a W5500 Ethernet module.

Thanks much!
Robert



Code:
#include <avr/io.h>
#include <avr/interrupt.h>
#include <SPI.h>                   
#include <Ethernet.h>

#define UDP_TX_PACKET_MAX_SIZE 1000  //increase UDP buffer size

////// Begin Ethernet Stuff //////

  IPAddress UDPServer(255, 255, 255, 255);    // Use UDP Broadcast address so the IP address of the server doesn't have to be known. Works as long as the devices are on the same LAN as the server.
  unsigned int UDPPort = 7772;                // local port to listen on
  char packetBuffer[UDP_TX_PACKET_MAX_SIZE];  // buffer to hold incoming packet,
  String UDPMessage;
  char MessageBuffer[100];                    // a string to send back
  EthernetUDP Udp;                            // An EthernetUDP instance to let us send and receive packets over UDP
  const String DeviceName = "IO_Test";
  byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; // MAC address and IP address for your controller

////// END Ethernet Stuff //////


// Timers 
IntervalTimer KeepAlive;                  // Sends a small network package periodically to keep the connection alive
const long KeepAliveFreq = 2000000;    // Keep Alive Frequency (2000000 = 2 sec)

void InitializeComms()
{
    // Open serial communications and wait for port to open:
  Serial.begin(9600);
  // this check is only needed on the Leonardo:
  
   // IMPORTANT!!!! Comment out for standalone UDP, Uncomment for tethered serial testing

  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  // start the Ethernet connection:
  Serial.println("Initialize Ethernet with DHCP:");
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for (;;)
      ;
  }

  Udp.begin(UDPPort);

  Serial.print("  IP Address ");
  Serial.println(Ethernet.localIP());
}


void KeepAlive_tick()
{
  SendUDPMessage(DeviceName + "|" + "Server" + "|" + "KeepAlive" + "|" + DeviceName); // Sends a small datagram to keep the network connection alive
}

void setup()
{
  InitializeComms(); // Initialize the serial port and the UDP Connection  
  KeepAlive.begin(KeepAlive_tick, KeepAliveFreq); // Start KeepAlive
}


void SendUDPMessage(String message)
{
  // Send UDP Message
  message.toCharArray(MessageBuffer,100); // 
  Udp.beginPacket(UDPServer, UDPPort);
  Udp.write(MessageBuffer);
  Udp.endPacket();

  Serial.println(message); // Transmit message serially
}

void loop()
{
  int packetSize = Udp.parsePacket(); // if there's data available, read a packet

  if (packetSize)
  {
    Serial.print("Packet Size: "); Serial.print(packetSize); 

    Udp.read(packetBuffer, UDP_TX_PACKET_MAX_SIZE); // read the packet into packetBufffer
    UDPMessage = packetBuffer;
  
    Serial.print("   Message: "); Serial.println(packetBuffer);
    memset(packetBuffer, 0, sizeof(packetBuffer));
  }
  delay(10);
}
 
Last edited:
Any thoughts on this? Although the timing varies, at some point the Udp.parsePacket() just completely stops working, but the UDP outputs I have on a interval timer (keep alive) are still going strong. Any idea why the incoming UDP randomly hangs at the following line?

int packetSize = Udp.parsePacket();

The reason I believe the problem with the above line of code, is if I use the following traps... every time it hangs up, I get "Trap 1", but I don't get "Trap 2", which is immediately the next line of code.

Serial.println("Trap 1");
int packetSize = Udp.parsePacket(); // if there's data available, read a packet
Serial.println("Trap 2");

Thanks,
Robert
 
Interesting. I'm having a very similar problem with Udp.parsePacket(); but over wifi using the WiFiNINA library.

Just as you have experienced, after a seemingly random (2min, 8min, 15min - never more than 40min) amount of time, UDP stops listening. I tried adding those two traps into my current sketch (see attached) and it died before arriving at "Trap 2." Last test lasted 21.7 min, including a 10ms delay in the loop.

I'm using a Teensy 3.6 and Adafruit AirLift ESP32 WiFi co-processor. Nothing else connected right now. The attached sketch has very few modifications from one of the examples included with the Adafruit fork of the WiFiNINA library. However, I've used half a dozen variations to get UDP to the teensy and all of them have shown this same problem.

Code:
/*
  WiFi UDP Send and Receive String

 This sketch wait an UDP packet on localPort using the WiFi module.
 When a packet is received an Acknowledge packet is sent to the client on port remotePort

 created 30 December 2012
 by dlf (Metodo2 srl)

 */


#include <SPI.h>
#include <WiFiNINA.h>

// Configure the pins used for the ESP32 connection
#if defined(TEENSYDUINO) 
  #define SPIWIFI      SPI  // The SPI port
  #define SPIWIFI_SS     10   // Chip select pin
  #define ESP32_RESETN   6    // Reset pin
  #define SPIWIFI_ACK    9   // a.k.a BUSY or READY pin
  #define ESP32_GPIO0   -1
#endif

#include <WiFiUdp.h>

int status = WL_IDLE_STATUS;
#include "arduino_secrets.h" 
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
char ssid[] = SECRET_SSID;        // your network SSID (name)
char pass[] = SECRET_PASS;    // your network password (use for WPA, or use as key for WEP)
int keyIndex = 0;            // your network key Index number (needed only for WEP)

unsigned int localPort = 2390;      // local port to listen on

char packetBuffer[255]; //buffer to hold incoming packet
char  ReplyBuffer[] = "acknowledged";       // a string to send back

WiFiUDP Udp;

void setup() {
  //Initialize serial and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  WiFi.setPins(SPIWIFI_SS, SPIWIFI_ACK, ESP32_RESETN, ESP32_GPIO0, &SPIWIFI);
  // check for the WiFi module:
  if (WiFi.status() == WL_NO_MODULE) {
    Serial.println("Communication with WiFi module failed!");
    // don't continue
    while (true);
  }

  String fv = WiFi.firmwareVersion();
  if (fv < "1.0.0") {
    Serial.println("Please upgrade the firmware");
  }

  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
    status = WiFi.begin(ssid, pass);

    // wait 10 seconds for connection:
    delay(10000);
  }
  Serial.println("Connected to wifi");
  printWifiStatus();

  Serial.println("\nStarting connection to server...");
  // if you get a connection, report back via serial:

  // disable low power mode - critical for low latency operation
  WiFi.noLowPowerMode();
  
  Udp.begin(localPort);
}

void loop() {
  unsigned long currentMillis = millis();
  // if there's data available, read a packet
  Serial.println("Trap 1");
  int packetSize = Udp.parsePacket();
  Serial.println("Trap 2");
  if (packetSize) {
    Serial.print("Received packet of size ");
    Serial.println(packetSize);
    Serial.print("From ");
    IPAddress remoteIp = Udp.remoteIP();
    Serial.print(remoteIp);
    Serial.print(", port ");
    Serial.println(Udp.remotePort());

    // read the packet into packetBufffer
    int len = Udp.read(packetBuffer, 255);
    if (len > 0) {
      packetBuffer[len] = 0;
    }
    Serial.println("Contents:");
    Serial.println(packetBuffer);

    Serial.print("time: ");
    Serial.println(currentMillis);

    // no need for replies
    /*
    // send a reply, to the IP address and port that sent us the packet we received
    Udp.beginPacket(Udp.remoteIP(), Udp.remotePort());
    Udp.write(ReplyBuffer);
    Udp.endPacket();
    */
  }
}


void printWifiStatus() {
  // print the SSID of the network you're attached to:
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // print your board's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("signal strength (RSSI):");
  Serial.print(rssi);
  Serial.println(" dBm");
}


Just for good measure, here are a few things I've tried:

* The same lock up happens on a Teensy LC and a different Airlift -- so I doubt it is a problem with connections or faulty hardware.

* I've tried a 5VDC power supply that provides 1 amp and it has no effect. The docs mention 250mA spikes for that shield, but a 1 amp supply should be able to handle this plus the teensy with no problem. So it seems the lock up still happens regardless of available power. I took the temp of the coprocessor and its about 91-94 deg F. Not sure if that is overly hot or not. Also tried testing without noLowPowerMode() called -- still locked up and the latency was just terrible.

* I've also tried delay(10) etc and interval timing to slow down that main loop. Still crashes after a seemingly random period of minutes.

Here is the bash script I'm using to send the UDP test strings:

Code:
for i in {1..10000}
do
  echo "/s_new" > /dev/udp/192.168.1.35/2390
  sleep .3s
  echo "/n_free" > /dev/udp/192.168.1.35/2390
  sleep .3s
done

The fact that we're both seeing this same problem on Ethernet and Wifi is pretty interesting. I mean, your Udp.parsePacket() and my Udp.parsePacket() are in two different libraries, are they not?

I know this was a while ago, maybe you solved the issue. If so, I'd love to hear what you did.

Thanks!
 
The EthernetUdp.cpp file has a note about the read failing.
On my test the _remaining count goes quite high (50K, perhaps I am hitting is with a lot), and for some reason the read(buf, count) then fails, the _remaining is not decremented and the result is an infinite loop.

This is my current, two hour old, work around (no root cause known):


--- arduino-1.8.12/hardware/teensy/avr/libraries/Ethernet/src/EthernetUdp.cpp 2020-05-29 23:28:52.056460912 -0700
+++ arduino-1.8.13/hardware/teensy/avr/libraries/Ethernet/src/EthernetUdp.cpp 2021-01-02 00:46:19.610552762 -0800
@@ -103,7 +103,10 @@
// could this fail (loop endlessly) if _remaining > 0 and recv in read fails?
// should only occur if recv fails after telling us the data is there, lets
// hope the w5100 always behaves :)
- read((uint8_t *)NULL, _remaining);
+ if (read((uint8_t *)NULL, _remaining) < 0) {
+ _remaining = 0;
+ break;
+ }
}

if (Ethernet.socketRecvAvailable(sockindex) > 0) {
 
Any updates on your work around? Did you manage to get it working for longer periods of time?

I have not fiddled with that project in a while and am afraid that I don't remember any progress since that work around or even how well it functioned.
 
I got frustrated and started trying out other hardware. No matter what I couldn't make the AirLift work for receiving UDP properly. I ended up moving backward to this:
https://www.adafruit.com/product/2999
It is older and more expensive, but it works with essentially the same approach and code, just calling a different library.
AdaFruit's response was unfortunately, that they've had other people mention it but most of their users and sending, and not receiving, UDP packets so it isn't a high priority for them.
 
Update:

After I removed: Udp.beginPacket(), Udp.write(), Udp.endPacket() - It seems that parsePacket() is working properly - tested for a couple of hours,
But whenever these functions are back in the code (even with a delay(300)) parsePacket() equals to 0 again, and stuck in an endless loop.
Tested both with QNEthernet and vjmuzik's NativeEthernet libraries.
 
When using QNEthernet: What are you seeing for the return value of endPacket()? Also beginPacket() and write()? For write(), does the return value match the number of bytes sent?
 
Hi shawn,

Return values:
beginPacket() = 1
endPacket() = 1
write() - returns the correct value of bytes
 
You’re saying that parsePacket() never returns non-zero when you send packets? Could you provide a simple example program that you know exhibits the problem?
 
My setup is 2 teensy's 4.1 Connected to each other with Ethernet cable via switch:
One is sending Udp.write("1"); with a delay of 100.
While the other one read incoming data and reply "1.

code of the receiving & replying Teensy:

#include <QNEthernet.h>

using namespace qindesign::network;

#define ARRAY_SIZE 10

char data2[ARRAY_SIZE]; //Read vector

//Local Unit network configuration
byte mac[] = {0x04, 0xE9, 0xE5, 0x0B, 0xFC, 0xD1 };
IPAddress staticIP(10, 100, 102, 127);
IPAddress gateway(10, 100, 102, 1);
IPAddress subnetMask(255, 255, 255, 0);
unsigned int sourcePort = 8002;

//Remote Unit network information
IPAddress destinationIp(10, 100, 102, 126);
unsigned int destinationPort = 8000;

EthernetUDP Udp;

void setup()
{
Ethernet.begin(staticIP, subnetMask, gateway);
Udp.begin(sourcePort);

Serial.begin(9600); //enable serial data print if needed
}

void loop()
{
int packetSize = Udp.parsePacket();
Serial.println(packetSize);
if (packetSize)
{
IPAddress remote = Udp.remoteIP();
Udp.read(data2,ARRAY_SIZE);
Serial.println(data2); //Incoming Data
delay(10);
Udp.beginPacket(destinationIp, destinationPort);
Udp.write("1");
Udp.endPacket();
}
delay(10);
}


Outcome: after a minute or two parsePacket = 0 and stays 0.
It seems that even if I change the delay it still happens (tried various values between 0-500).
When I remove the lines:
Udp.beginPacket(destinationIp, destinationPort);
Udp.write("1");
Udp.endPacket();

Everything is working (tested for 16 hours straight), parsePacket varies from 1 to 0 but the program keeps running.
 
One major thing I noticed about the code is that you're printing strings without a guaranteed NUL terminator. `Udp.write("1")` only sends one byte. Then, when it's received into `data2` on the other Teensy, not only is the received size from `Udp.read()` not checked, but there's no guarantee that `data2[1]` contains a NUL character, so calling `Serial.println(data2)` might not be well-behaved, depending what's in memory.

Some other notes:
1. There's no need to declare your own MAC address. (But having it there won't do anything.)
2. It might not make a difference because `Serial` goes over USB, but I'm in the habit of using 115200 and not 9600.
3. It's better to check `udp.parsePacket()` without a delay. I would restructure the code to only limit sending rate and not limit receiving rate. i.e. use `elapsedMillis` or some math with `millis()` instead of using `delay()` at all.

First make these changes and then see if that helps. I haven't ruled out the possibility of a problem, but I want to make sure the problem isn't because of the potential NUL issue.

Question: Is the same code running on both Teensys? At the top you say one is sending and one is receiving, which implies the code is different on each, but then you also say that the example program is running on both.
 
Ok thanks for the tips! I'll give it a try.

Regarding your question, no:
Teensy 1: Sending a UDP message containing "1" at 10 Hz
Teensy 2: Receive and send (the attached code)

Until I finish do you have an example of two teensy's sending and receiving messages using UDP?
 
Here's my quickie test programs. I've never seen it fail on the receiving Teensy. I always see "size=1" printed.
Question: Once you see the packet size change to zero, does it ever go back to being non-zero?

Teensy:
Code:
#include <QNEthernet.h>

using namespace qindesign::network;

// Port on which to send and receive
constexpr uint16_t kPort = 8000;
IPAddress destIP{10, 100, 102, 126};
//IPAddress staticIP{10, 100, 102, 127};
//IPAddress netmask{255, 0, 0, 0};
//IPAddress gateway{10, 100, 102, 1};

EthernetUDP udp;
constexpr int kBufSize = Ethernet.mtu() - 8 - 20;
uint8_t buf[kBufSize];

constexpr uint8_t kSendContents[]{'1'};

int sentPacketCount = 0;
int recvPacketCount = 0;
elapsedMillis sendTimer;

void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 4000) {
    // Wait for Serial
  }
  stdPrint = &Serial;  // Make printf work (A QNEthernet feature)

  printf("Starting Ethernet with DHCP...\n");
  if (!Ethernet.begin()) {
    printf("Error initializing Ethernet\n");
    return;
  }
  if (!Ethernet.waitForLocalIP(10'000)) {
    printf("Error getting IP address via DHCP\n");
    return;
  }

  // If using a static IP, comment out the above and uncomment this:
//  printf("Starting Ethernet with static IP...\n");
//  if (!Ethernet.begin(staticIP, netmask, gateway)) {
//    printf("Error initializing Ethernet\n");
//    return;
//  }
//  if (!Ethernet.waitForLink(10'000)) {
//    printf("Error getting link\n");
//    return;
//  }
  
  IPAddress ip = Ethernet.localIP();
  printf("    Local IP    = %u.%u.%u.%u\n", ip[0], ip[1], ip[2], ip[3]);
  ip = Ethernet.subnetMask();
  printf("    Subnet mask = %u.%u.%u.%u\n", ip[0], ip[1], ip[2], ip[3]);
  ip = Ethernet.gatewayIP();
  printf("    Gateway     = %u.%u.%u.%u\n", ip[0], ip[1], ip[2], ip[3]);
  ip = Ethernet.dnsServerIP();
  printf("    DNS         = %u.%u.%u.%u\n", ip[0], ip[1], ip[2], ip[3]);

  // Start listening on port kPort
  udp.begin(kPort);
}

void loop() {
  int size = udp.parsePacket();
  if (size > 0) {
    printf("%d: size=%d\n", ++recvPacketCount, size);
    if (size > kBufSize) {
      size = kBufSize;
    }
    udp.read(buf, size);
    // Do stuff with the buffer
    // If we don't do anything with it, we don't have to read it
  }

  // Send at about 10Hz (1000(ms/s)/10Hz = 100ms)
  if (sendTimer >= 100) {
    udp.beginPacket(destIP, kPort);
    udp.write(kSendContents, sizeof(kSendContents));
    if (!udp.endPacket()) {
      printf("Failed to send\n");
    } else {
      ++sentPacketCount;
//      printf("%d: Sent\n", sentPacketCount);
    }
    sendTimer = 0;
  }
}

To generate packets, I made a Processing sketch that runs on the same network:
Code:
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;

// Teensy address and port
static final String ADDR = "10.100.102.127";
static final int PORT = 8000;

DatagramSocket socket;
DatagramPacket packet;
InetAddress addr;
byte[] data = new byte[]{'1'};

int packetCount = 0;

void setup() {
  try {
    socket = new DatagramSocket();
    packet = new DatagramPacket(data, data.length,
                                InetAddress.getByName(ADDR), PORT);
    println("Starting...");
  } catch (IOException ex) {
    println(ex);
  }
}

void draw() {
  try {
    socket.send(packet);
    ++packetCount;
    //println("Sent " + packetCount);
  } catch (IOException ex) {
    println(ex);
  }

  delay(100);  // <-- Change this to change frequency
}

Just because I haven't seen it fail, doesn't mean there isn't some subtle problem. Let me know if my code above makes it work. If it doesn't work, the problem may be something about your network.

If my code does work, maybe it's something about how you've coded it. In this case, could you change one thing at a time to see what makes your code work when transforming it into something similar to my code?
 
Back
Top