New lwIP-based Ethernet library for Teensy 4.1

[shawn]

Hello, all. I went ahead and made an lwIP-based Ethernet library for the Teensy 4.1. Some details about it:

• This currently is not a complete drop-in replacement for the Arduino-style Ethernet API. (But it’s reasonably close.) The README details the differences. I haven't yet decided the direction I want to take, because I don't love the global variable-style approach, plus there are some things that are underspecified. This may end up being something completely different, or it may be similar; I haven't decided yet. Heck, maybe I look at how mbed does it...
• The polling function is set up internally via EventResponder to run inside yield(); no timers are used.
• I haven't tested it with the Arduino IDE; it probably won't work. You need to use PlatformIO for now.
• Use QNEthernet.h as the main include file.
• I use namespaces.
• lwIP version is 2.1.2.
• main.cpp is a hodge-podge testing playground from which you can glean information about how to use the API. It's very similar to the Arduino API. You'll note mDNS and an OSC parser, useful with the TouchOSC app — try it. Also note that you don't actually need to include any of the lwip headers; I'm just using them for testing inside some callbacks.

My current to do list:

• Tune lwIP. I could use some help with this. (@manitou, I know you've already done some tuning; I point to this in the README.)
• See if there's a way to improve DHCP response time. In my testing, it usually takes about 10-12 seconds to get an IP address.
• Decide how to restructure the internals, either for something different or to make it even more compatible with the Arduino API. For example, right now, EthernetClient is not copyable, only movable, having to do with how I made the insides mostly client-managed instead of server-managed.
• I haven't settled on a name.

I got this to a point where all my basic testing works. This project is still very much a work in progress, though, aren't they all?

Last point for now: Thank you CrashReporter and everyone who was involved with making that work!

Here's the link: https://github.com/ssilverman/QNEthernet

 

[shawn]

Version 0.3.0 (tagged as "v0.3.0" in the repo) has:

• Revamped and centralized connection management.
• Works as an Arduino library.
• Bug and kink fixes.
• Update to "it's really v2.1.2 this time" lwIP.

Link: https://github.com/ssilverman/QNEthernet/releases/tag/v0.3.0

I'm working on a "How To", but here's a few things you can do to adapt your code:

1. Change #include <Ethernet.h> to #include <QNEthernet.h>. Note that this include already includes the header for EthernetUDP, so you can remove any #include <EthernetUdp.h>.
2. Just below that, add: using namespace qindesign::network;
3. You likely don't want or need to set/choose your own MAC address, so just call Ethernet.begin() with no arguments. This version uses DHCP. The three-argument version (IP, subnet mask, gateway) sets those parameters instead of using DHCP. If you really want to set your own MAC address, for now, consult the code.
4. It may take 10-15 seconds to get a DHCP address (or whatever it is), so wait for a little bit until Ethernet.localIP() isn't INADDR_NONE. You could use an elapsedMillis with delays of, say, 10ms, until there's an address or the elapsed time reaches a maximum (eg. 15000ms).
5. Ethernet.hardwareStatus() always returns zero and Ethernet.linkStatus() returns a bool (i.e. not that EthernetLinkStatus enum).
6. Most other things should be the same.

 

[Frank B]

This is great.

 

[shawn]

I just released v0.4.0. The changes:

• Updated to lwIP v2.1.3-rc1.
• Moved the global objects (Ethernet and MDNS) into the same namespace as everything else. This means the "using namespace" approach is probably the easiest option.
• Fixed UDP multicast: it wasn't joining the IGMP group or checking the address correctly (byte ordering issue).
• Can now add TXT record items to mDNS services.
• New Ethernet::waitForLocalIP(timeout) function to make it easier to wait for DHCP-assigned addresses.
• Added the ability to re-announce mDNS services. I'm not certain if it's supposed happen, but the services disappear after around the TTL duration with no refresh. I don't know if this is my fault for not understanding something, or a limitation in the stack.
• Updated the README with new instructions and notes.

https://github.com/ssilverman/QNEthernet/releases/tag/v0.4.0

 

[khoih-prog]

This is very good. I've modified and tested with several examples, and they are working smoothly with DHCP. Static IP still not working yet, will fix later.

I've created a PR for the library to add several examples, so that it's much easier for anyone to start using the library.

Add example for QNEthernet #1

 

[shawn]

Thank you for these examples. I'll see what's up with static IP...

Update: I just tested static IP and it seems to work fine. But now I'm just realizing that you may have meant the examples and not necessarily the main code...

 

[joepasquariello]

I just released v0.4.0. The changes:

shawn, thanks very much for this work. I have a question about loopServer(). It contains the clause shown below (printf was my addition). When the program starts, all elements of clients[] are 0, so this loop prints "Client X stop" for all 8 clients, on every pass. What is this code meant to do? If (!clients) is true, i.e (clients==0), what does it mean to call clients.stop()? Seems like something is missing.

  // stop any clients which disconnect
  for (int i = 0; i < 8; i++) {
    if (!clients[i]) {
      clients[i].stop();
      Serial.printf("Client %d stop()\n", i);
    }
  }

EDIT: I'm using the FNET utility "fbench.exe" to connect and send packets to the T41. I assume this application is disconnecting after sending the specified number of packets, but the server application in the T41 never reports a disconnect. The result is that I can connect/send until all of the clients[] have been used, and then the server application will no longer accept connections.

 

[shawn]

Hi, Joe. That test code in test/main.cpp was adapted from all the Arduino test code here: https://www.arduino.cc/en/Reference/Ethernet

I just wanted to make sure the library worked as expected with existing code, merely as a starting point. I'm not the fondest of how the example code is structured (or most Arduino-style code, for that matter) and it is not how I'd write my own network programs or examples. You can consider this code obsolete and not a good example. Maybe I should remove it if it's causing confusion.

At some point in the near future, I'll be adding better examples.

 

[shawn]

Joe, I'll add another point: The code feels wrong because why would you stop a client whose Boolean value returns false? In fact, a client returning a Boolean isn't specified very well and is actually contradictory in the examples from that Arduino link above (https://www.arduino.cc/en/Reference/Ethernet). I've chosen to return `true` if connected and `false` otherwise; I'm not sure how other libraries do it, or how it's even supposed to be done, other than the fact that it exists in the Arduino API.

In summary, I don't like that code and it really should be `if (clients)` and not `if (!clients)`, per how my library works.

 

[shawn]

Joe, one more thing: The original example actually has `clients && !clients.connected()`, and I think I was experimenting, but that code is confusing in any case. I think they may have meant "active and no data pending". (Same for this line: `while (clients && clients.available() > 0)`)

In any case, don't use the code in `test/main.cpp`.

 

[joepasquariello]

At some point in the near future, I'll be adding better examples.

Thanks, Shawn. I started with the TCP server example because that's one of my use cases. I'm pretty much an Ethernet beginner, but I'll try to figure out (google?) how lwip detects and handles a TCP client disconnect.

 

[shawn]

Joe, can you tell me more about your use case? Different uses demand different styles of API usage and state maintenance. For example, an HTTP server could be done by having itself manage all the connections (eg. accept()-style), but if you choose to use that "server.available()" approach, the HTTP server would instead need to manage state a certain way.

[Note: I updated test/main.cpp with some notes and a slightly changed loopServer().]

For client disconnects, lwIP sends an OK and a null pbuf to the receiver callback (I’m using callbacks internally; there’s an option to not use them). QNEthernet handles this case and disconnects the client state, but also stores remaining data in the case of errors.

 

[joepasquariello]

Joe, can you tell me more about your use case?

My use case is a simple command/control interface via TCP. The device (T41) would listen/accept connections, then read, process, and reply to commands and data requests from those connections. The only thing missing from the loopServer example is to detect and clean up when a client disconnects, so that connection "slot" is available for another client.

 

[shawn]

Are you always expecting a fixed number of bytes or an arbitrary number of bytes (eg. controlled by, say, a “length” field or two)?

 

[shawn]

One of the things I don’t like about the “server.available()” API is that it’s not complete. There’s no defined way to map the client object you receive to state being managed by your program. The object itself may be a copy, or it may be “moved” (in the C++ sense).

Same goes with storing received client objects from “server.accept()”. There’s copies vs. references vs. “moves” to worry about, and there’s a few gotchas in C++ if these aren’t handled properly.

Maybe I’ll add something… EthernetClient::id() perhaps?

 

[joepasquariello]

The packet content and number of fields is arbitrary. Right now, I'm not trying to parse packets or build a real application. I'm just trying to test the capability to accept connections from clients, echo packets back to the client(s) for as long as they are active, and detect when they disconnect. Is it possible for the server to detect when a client disconnects, or is a timeout on receive the best one can do? I have to say I'm very confused about the Arduino approach to Ethernet, as opposed to directly using a socket API.

 

[shawn]

Here's a survey of how connections (aka "EthernetClient") work (at least with QNEthernet):

1. `connected()`: Returns whether connected OR data is still available (or both)
2. `operator bool`: Returns whether connected (at least in QNEthernet)
3. `available()`: Returns the amount data is available, whether the connection is closed or not
4. `read`: Reads data if there's data available, whether the connection's closed or not

Connections will be closed automatically if the client shuts down a connection, and QNEthernet will properly handle the state such that the API behaves as expected. In addition, if a client closes a connection, any buffered data will still be available via the client API. If it were up to me, I'd have swapped the meaning of "operator bool" and "connected()", but see the above list as a guide.

Some options:

1. Keep checking `connected()` (or "operator bool") and `available()`/`read` to keep reading data. The data will run out when the connection is closed and after all the buffers are empty. The calls to `connected()` (or "operator bool") will indicate connection status (plus data available in the case of `connected()` or just connection state in the case of "operator bool").
2. Same as the above, but without one of the two connection-status calls (`connected()` or "operator bool"). The data will just run out after connection-closed and after the buffers are empty.

Does this help clarify things?

 

[joepasquariello]

Yes, that is helpful. I didn't realize that "clients" and "!clients" were uses of "operator bool", as opposed to testing for value 0.

EDIT: I'm finding that the example program does mostly what I expected if I simply replace read() with read(buf,size) and replace Serial.write() with Serial.printf(nbytes read). When I do that, I get reasonable behavior with respect to elements of clients[] being "re-used".

Thanks for all your help.

 

[shawn]

Here's a quickie example of how a server could process a continuous stream of fixed-width messages from multiple clients:

// SPDX-FileCopyrightText: (c) 2021 Shawn Silverman <shawn@pobox.com>
// SPDX-License-Identifier: MIT

// FixedWidthServer demonstrates how to serve a protocol having a continuous
// stream of fixed-size messages from multiple clients.
// This file is part of the QNEthernet library.

// C++ includes
#include <algorithm>
#include <utility>
#include <vector>

#include <QNEthernet.h>

using namespace qindesign::network;

constexpr uint32_t kDHCPTimeout = 10000;  // 10 seconds
constexpr uint16_t kServerPort = 5000;
constexpr int kMessageSize = 10;  // Pretend the protocol specifies 10 bytes

// Keeps track of state for a single client.
struct ClientState {
  ClientState(EthernetClient client)
      : client(std::move(client)) {}

  EthernetClient client;
  int bufSize = 0;  // Keeps track of how many bytes have been read
  uint8_t buf[kMessageSize];
  bool closed = false;
};

// Keeps track of what and where belong to whom.
std::vector<ClientState> clients;

// The server.
EthernetServer server{kServerPort};

void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 4000) {
    // Wait for Serial to initialize
  }
  Serial.println("Starting...");

  uint8_t mac[6];
  Ethernet.macAddress(mac);
  Serial.printf("MAC = %02x:%02x:%02x:%02x:%02x:%02x\n",
                mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);

  Serial.println("Starting Ethernet with DHCP...");
  Ethernet.begin();
  Ethernet.waitForLocalIP(kDHCPTimeout);
  if (Ethernet.localIP() == INADDR_NONE) {
    Serial.println("Failed to get IP address from DHCP");
  } else {
    IPAddress ip = Ethernet.localIP();
    Serial.printf("    Local IP    = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
    ip = Ethernet.subnetMask();
    Serial.printf("    Subnet mask = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
    ip = Ethernet.gatewayIP();
    Serial.printf("    Gateway     = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
    ip = Ethernet.dnsServerIP();
    Serial.printf("    DNS         = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
  }

  // Start the server
  Serial.printf("Listening for clients on port %u...\n", kServerPort);
  server.begin();
}

// Process one message. This implementation simply prints to Serial.
//
// We could pass just the buffer, but we're passing the whole state here so
// you know which client it's from.
void processMessage(ClientState &state) {
  Serial.print("Message: ");
  Serial.write(state.buf, kMessageSize);
  Serial.println();
}

void loop() {
  EthernetClient client = server.accept();
  if (client) {
    Serial.print("Client connected: ");
    Serial.println(client.remoteIP());
    clients.emplace_back(std::move(client));
    Serial.printf("Client count: %u\n", clients.size());
  }

  // Process data from each client
  for (ClientState &state : clients) {  // Use a reference
    if (!state.client.connected()) {
      state.closed = true;
      continue;
    }

    int avail = state.client.available();
    if (avail > 0) {
      int toRead = std::min(kMessageSize - state.bufSize, avail);
      state.bufSize += state.client.read(&state.buf[state.bufSize], toRead);
      if (state.bufSize >= kMessageSize) {
        processMessage(state);
        state.bufSize = 0;
      }
    }
  }

  // Clean up all the closed clients
  size_t size = clients.size();
  clients.erase(std::remove_if(clients.begin(), clients.end(),
                               [](const auto &state) { return state.closed; }),
                clients.end());
  if (clients.size() != size) {
    Serial.printf("Client count: %u\n", clients.size());
  }
}

I might write another example that does arbitrary size, but for now, you could extrapolate from here.

 

[joepasquariello]

Thanks, Shawn. I can see that you're using an STL vector, adding clients on connect and removing on disconnect, so you only iterate over clients that are connected. I'll do something like that.

 

[khoih-prog]

Thank you for these examples. I'll see what's up with static IP...

Update: I just tested static IP and it seems to work fine. But now I'm just realizing that you may have meant the examples and not necessarily the main code...

The static IP issue in the examples has been fixed.

The call to Ethernet.setDNSServerIP(mydnsServer) must be placed after Ethernet.begin(myIP, myNetmask, myGW) for the DNS server IP to be valid. Otherwise, EthernetClient::connect() will always return false

 

[khoih-prog]

EthernetWebServer Library now support QNEthernet on Teensy 4.1.

A new release will be published in this weekend

 

[khoih-prog]

EthernetWebServer v1.6.0 has been released to support this QNEthernet library.

The following is debug terminal output when running example MQTTClient_Auth on Teensy 4.1 using QNEthernet Library

Start MQTTClient_Auth on TEENSY 4.1 using QNEthernet
EthernetWebServer v1.6.0
[EWS] =========== USE_QN_ETHERNET ===========
Initialize Ethernet using static IP => IP Address = 192.168.2.222
Attempting MQTT connection to broker.emqx.io...connected
Message Send : MQTT_Pub => Hello from MQTTClient_Auth on TEENSY 4.1 using QNEthernet
Message arrived [MQTT_Pub] Hello from MQTTClient_Auth on TEENSY 4.1 using QNEthernet
Message Send : MQTT_Pub => Hello from MQTTClient_Auth on TEENSY 4.1 using QNEthernet
Message arrived [MQTT_Pub] Hello from MQTTClient_Auth on TEENSY 4.1 using QNEthernet

 

[khoih-prog]

The QNEthernet, using lwip, is really a great library, and much better than the other alternative. Adding its support to other libraries will be very easy and smooth.


EthernetWebServer_SSL Library now supports QNEthernet on Teensy 4.1





The following is debug terminal output when running example MQTTClient_SSL on Teensy 4.1 using QNEthernet Library

Starting MQTTClient_SSL on TEENSY 4.1 using QNEthernet
EthernetWebServer_SSL v1.6.0
[ETHERNET_WEBSERVER] =========== USE_QN_ETHERNET ===========
Initialize Ethernet using static IP => IP Address = 192.168.2.222
Attempting MQTTS connection to broker.emqx.io...connected
Message Send : MQTT_Pub => Hello from MQTTClient_SSL on TEENSY 4.1
Message arrived [MQTT_Pub] Hello from MQTTClient_SSL on TEENSY 4.1
Message Send : MQTT_Pub => Hello from MQTTClient_SSL on TEENSY 4.1
Message arrived [MQTT_Pub] Hello from MQTTClient_SSL on TEENSY 4.1
Message Send : MQTT_Pub => Hello from MQTTClient_SSL on TEENSY 4.1
Message arrived [MQTT_Pub] Hello from MQTTClient_SSL on TEENSY 4.1

 

[shawn]

Here's a quickie example I'm calling "LengthWidthServer" that shows how a server can process a continuous stream of messages from multiple clients, where each message has a length byte:

// SPDX-FileCopyrightText: (c) 2021 Shawn Silverman <shawn@pobox.com>
// SPDX-License-Identifier: MIT

// LengthWidthServer demonstrates how to serve a protocol having a continuous
// stream of messages from multiple clients, where each message starts with a
// one-byte length field. This is similar to the FixedWidthServer example, but
// the data stream indicates the size of each message. It shows a simple
// version of how to use states when parsing, when the position in the client
// stream is arbitrary.
// 
// This file is part of the QNEthernet library.

// C++ includes
#include <algorithm>
#include <utility>
#include <vector>

#include <QNEthernet.h>

using namespace qindesign::network;

constexpr uint32_t kDHCPTimeout = 10000;  // 10 seconds
constexpr uint16_t kServerPort = 5000;

// Where are we with message parsing?
enum class MessageParseState {
  kStart,  // Starting state
  kValue,  // Reading the value
};

// Keeps track of state for a single client.
struct ClientState {
  ClientState(EthernetClient client)
      : client(std::move(client)) {
    reset();
  }

  EthernetClient client;
  bool closed = false;

  MessageParseState parseState;
  int messageSize;   // The current message size
  int bufSize;       // Keeps track of how many bytes have been read
  uint8_t buf[255];  // Do the easy thing and allocate the maximum possible

  // Reset the client state.
  void reset() {
    messageSize = 0;
    bufSize = 0;
    parseState = MessageParseState::kStart;
  }
};

// Keeps track of what and where belong to whom.
std::vector<ClientState> clients;

// The server.
EthernetServer server{kServerPort};

void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 4000) {
    // Wait for Serial to initialize
  }
  Serial.println("Starting...");

  uint8_t mac[6];
  Ethernet.macAddress(mac);
  Serial.printf("MAC = %02x:%02x:%02x:%02x:%02x:%02x\n",
                mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);

  Serial.println("Starting Ethernet with DHCP...");
  Ethernet.begin();
  Ethernet.waitForLocalIP(kDHCPTimeout);
  if (Ethernet.localIP() == INADDR_NONE) {
    Serial.println("Failed to get IP address from DHCP");
  } else {
    IPAddress ip = Ethernet.localIP();
    Serial.printf("    Local IP    = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
    ip = Ethernet.subnetMask();
    Serial.printf("    Subnet mask = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
    ip = Ethernet.gatewayIP();
    Serial.printf("    Gateway     = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
    ip = Ethernet.dnsServerIP();
    Serial.printf("    DNS         = %u.%u.%u.%u\n",
                  ip[0], ip[1], ip[2], ip[3]);
  }

  // Start the server
  Serial.printf("Listening for clients on port %u...\n", kServerPort);
  server.begin();
}

// Process one message. This implementation simply prints to Serial, escaping
// some characters.
//
// We could pass just the buffer, but we're passing the whole state here so
// we know which client it's from.
void processMessage(const ClientState &state) {
  Serial.printf("Message [%d]: ", state.messageSize);
  for (int i = 0; i < state.messageSize; i++) {
    uint8_t b = state.buf[i];
    if (b < 0x20) {
      switch (b) {
        case '\a': Serial.print("\\q"); break;
        case '\b': Serial.print("\\b"); break;
        case '\t': Serial.print("\\t"); break;
        case '\n': Serial.print("\\n"); break;
        case '\v': Serial.print("\\v"); break;
        case '\f': Serial.print("\\f"); break;
        case '\r': Serial.print("\\r"); break;
        case '\\': Serial.print("\\\\"); break;
        default:
          Serial.printf("\\x%x%x", (b >> 4) & 0x0f, b & 0x0f);
      }
    } else if (0x7f <= b && b < 0xa0) {
      Serial.printf("\\x%x%x", (b >> 4) & 0x0f, b & 0x0f);      
    } else {
      Serial.write(b);
    }
  }
  Serial.println();
}

void loop() {
  EthernetClient client = server.accept();
  if (client) {
    Serial.print("Client connected: ");
    Serial.println(client.remoteIP());
    clients.emplace_back(std::move(client));
    Serial.printf("Client count: %u\n", clients.size());
  }

  // Process data from each client
  for (ClientState &state : clients) {  // Use a reference
    if (!state.client.connected()) {
      state.closed = true;
      continue;
    }

    int avail = state.client.available();
    while (avail > 0) {
      switch (state.parseState) {
        case MessageParseState::kStart:
          state.messageSize = state.client.read();
          avail--;
          state.parseState = MessageParseState::kValue;
          break;

        case MessageParseState::kValue: {
          int read = std::min(state.messageSize - state.bufSize, avail);
          read = state.client.read(&state.buf[state.bufSize], read);
          state.bufSize += read;
          avail -= read;
          if (state.bufSize >= state.messageSize) {
            processMessage(state);
            state.messageSize = 0;
            state.bufSize = 0;
            state.parseState = MessageParseState::kStart;
          }
          break;
        }

        default:
          break;  // Shouldn't happen because we put in all the states
      }
    }
  }

  // Clean up all the closed clients
  size_t size = clients.size();
  clients.erase(std::remove_if(clients.begin(), clients.end(),
                               [](const auto &state) { return state.closed; }),
                clients.end());
  if (clients.size() != size) {
    Serial.printf("Client count: %u\n", clients.size());
  }
}