Inherent Latency in Ethernet Modbus over Serial with Teensy 4.1?

milesg

Member
Hi All,
I've been working on a project where the circuitry allows for either Ethernet or Serial connectivity based on a jumper position. The code is essentially identical between either mode of communication, the only difference being some ternary operators set up to make sure I'm addressing the correct server:

I'm using QNEthernet and ArduinoModbus libraries.
C++:
// Here's an example of how I'm getting data from the chosen connection to set my output:
  for (uint8_t i=0; i<outputChannelsNum; i++) {
    boolean state = (jumperMode==2)?modbusTCPServer.coilRead(i+1):ModbusRTUServer.coilRead(i+1); // Get target state from coils 1-24
    if (digitalRead(outputPins[i])!=state) digitalWrite(outputPins[i], state); // write new value if changed
  }

Everything works great! No issues with the circuit or function of the modbus server with the jumper set in either position. The concerning bit is that my code responds to modbus calls over Serial in about 25ms (that's at 9600 Baud), and about 240ms when using Ethernet.

Here's the loop I'm using:
C++:
void loop() {
  if (jumperMode==2) { // Ethernet loop
    qn::EthernetClient client = EthServer.available();
    if (client) {
      modbusTCPServer.accept(client);
      if (client.connected()) { // Examples shows while() here. This prevents the loop from recovering when the cable is disconnected. if() resolves the issue.
        if (modbusTCPServer.poll()) { // poll() returns true when a request is present
          if (modbusTCPServer.holdingRegisterRead(44)) saveDefaults();
          if (modbusTCPServer.holdingRegisterRead(45)) closeAll();
          if (modbusTCPServer.holdingRegisterRead(46)) FSVAll();
          updateOutput();
        }
      }
    }
  }

  if (jumperMode==3 && ModbusRTUServer.poll()) { // Serial loop
    if (ModbusRTUServer.holdingRegisterRead(44)) saveDefaults();
    if (ModbusRTUServer.holdingRegisterRead(45)) closeAll();
    if (ModbusRTUServer.holdingRegisterRead(46)) FSVAll();
    updateOutput();
  }
}

You can see that I've copied the example code nearly 1:1 (the only difference in structure is that I changed a 'while' to an 'if' as noted, maybe there's a better way to deal with the issue I had there.) Of course there are the extra things that need to be done to work with the Ethernet server, but the extra latency seems excessive.

Is this normal behavior, or am I doing something wrong? I believe I've posted the relevant code, but let me know if you want to see more details.
Thanks for any insight that can be given on this matter.
 
Last edited:
If you can share a complete, minimal, example that shows the issue, maybe I could have a look. Also describe how to test it.
 
Alright, so here's a simpler version:

It just sets up the server with only one coil and the ability to switch between communication modes based on the jumper position at startup (the jumper is made for configuration at the time of assembly, not changing while the code is running or anything fancy like that). I have a test PCB with a MAX3232 chip on the Uart port and the T4.1 Ethernet kit for the TCP comms. So I'm testing, where this device is plugged into my Gigabit router and PC's COM port. Not sure what the easiest way for someone else to test this might be, but I think the intent is pretty clear here. Note that for my PCB I edited RS485.cpp to use Serial5, and I've bumped up the serial speed to 19200, but See the screenshots below to see the results of running this code.

C++:
#include <QNEthernet.h> // Most optimized Ethernet library for Teensy 4.1 Seems to have resolved memory leak in other libraries
#include <ArduinoModbus.h> // Dependency ArduinoRS485\RS485.cpp must have line 209 edited to use Serial5 and associated pins.
#include <EEPROM.h> // Enable access to EEPROM on Teensy board
namespace qn = qindesign::network;

qn::EthernetServer EthServer(502);
ModbusTCPServer modbusTCPServer;

void updateOutput(); // Updates all output pins to high/low based on Modbus coil values

// Global variables
byte mac[] = { 0x9C, 0x52, 0x69, 0x62, 0x65, 0x72 }; // Default MAC address
byte ip[] = { 192, 168, 1, 50 }; // Default IP address
byte subnet[] = { 255, 255, 255, 0 }; // Default Subnet mask
uint8_t jumperMode = 0; // Set during setup() to select boot mode. 1=Load defaults, 2=Ethernet, 3=Serial

void setup() {
  for (uint8_t i=2; i<5; i++) { // Set up input pins
    pinMode(i, INPUT_PULLUP); // Set all 3 jumper positions as inputs
  }
  pinMode(13, OUTPUT);

  delay(1); // Wait for inputs to stabilize. Stops false jumperMode=1 on cold start

  // Set mode based on jumper position
  if (digitalRead(2)==LOW) {
    jumperMode=1; // unused in this example
  } else if (digitalRead(3)==LOW) { // Ethernet Connection
    jumperMode=2;
  } else if (digitalRead(4)==LOW) { // Serial Connection
    jumperMode=3;
  }

  if (!jumperMode) { // No jumper installed
    while(1); // Halt program
  }

  if (jumperMode==2) { // Ethernet mode
    qn::Ethernet.begin(mac, ip, subnet); // Create object
    EthServer.begin(); // Start listening for client connections
    modbusTCPServer.begin(); // Start modbus service
    modbusTCPServer.configureCoils(0, 1);
  }

  if (jumperMode==3) { // Serial mode
    ModbusRTUServer.begin(1, 19200); // Start modbus service
    ModbusRTUServer.configureCoils(0, 1); // Coils for turning all shutters off, and individual control of each channel
  }
}

void loop() {
  if (jumperMode==2) { // Ethernet loop
    qn::EthernetClient client = EthServer.available();
    if (client) {
      modbusTCPServer.accept(client);
      if (client.connected()) { // Examples shows while() here. This prevents the loop from recovering when the cable is disconnected. if() resolves the issue.
        if (modbusTCPServer.poll()) { // poll() returns true when a request is present
          updateOutput();
        }
      }
    }
  }

  if (jumperMode==3) { // Serial loop
    if (ModbusRTUServer.poll()){
      updateOutput();
    }
  }
}

void updateOutput() {
  boolean state=(jumperMode==2)?modbusTCPServer.coilRead(0):ModbusRTUServer.coilRead(0);
  digitalWrite(13, state);
}

In the attached images you can see how my Modbus testing software (QModMaster) is sending a request to read the value of the coil and the time it takes to get a response. The results are a bit faster in this simplified code, but still pretty slow on the ethernet front.

EDIT: made code blocks cpp for clarity. Also, I can ping the Teensy and always get <1ms response

Thanks for taking the time to look at this!
 

Attachments

  • SerialLatency.JPG
    SerialLatency.JPG
    30.4 KB · Views: 7
  • EthernetLatency.JPG
    EthernetLatency.JPG
    41.9 KB · Views: 8
Last edited:
A couple of things I noticed at first glance:
1. There's no need to specify a MAC address on the Teensy 4.1 because there's a built-in one. May I suggest using the begin(ip, netmask, gateway) function instead? As well, the one you're actually using in the code you pasted is begin(mac, ip, dns). See: https://www.arduino.cc/reference/en/libraries/ethernet/ethernet.begin/ (I don't love the Arduino Ethernet API for these things.) (This is the one you want if you really must specify your own MAC address: Ethernet.begin(mac, ip, dns, gateway, subnet))
2. I would use if (client.available() > 0) instead of if (client.connected()). I think that's the real intent. (You could even experiment with while (client.available() > 0) because that will be false when there's no data.) The Boolean check just checks if connected. The connected() function checks if connected OR if there's still data available, or both, so it will always be true as long as the connection exists OR there's data, or both. The available() function checks how much data is available, even after disconnect (similar to connected()).

This link may also help: https://github.com/ssilverman/QNEth...ce/README.md#how-to-write-data-to-connections
Basically, QNEthernet's client send functions don't send data until either a short timer expires, until there's enough data for a packet, or until flush() is called on the connection. It looks like the place where that Modbus library is calling write() is here: https://github.com/arduino-librarie...3a4f1331dc8/src/libmodbus/modbus-tcp.cpp#L202

Try changing to something like this:
C++:
size_t retval = ctx_tcp->client->write(req, req_length);
ctx_tcp->client->flush();
return retval;
(Note: I haven't tested this.)

Let me know if these tips help.
 
Last edited:
I’ll add: it’s not correct to always assume some write() call (on, say, the Arduino Print interface, which many types of objects implement) completes without buffering. Unfortunately, many programs and libraries assume this.
See:

Maybe I need to add some QNEthernet option that performs a flush() on every write() call? I’m hesitant to do this just yet, though, because it disagrees with how write() is supposed to work. But it may help to not have to modify some libraries that make this unfortunate assumption. It might also affect TCP throughput in some cases, though.
 
Last edited:
I just realized that all the bytes still may not get out. Any write() calls (with any library, really) should have their return values checked and then any unsent bytes should be re-sent. write() doesn't always send all the bytes it's given. A better QNEthernet equivalent would be:
C++:
size_t retval = ctx_tcp->client->writeFully(req, req_length);
ctx_tcp->client->flush();
return retval;

This uses writeFully() instead of write(). (You could accomplish the same thing yourself by looping until complete. See also: https://github.com/ssilverman/QNEth...ce/README.md#how-to-write-data-to-connections)

In summary, just using a write() call isn't correct usage, in most cases.
 
Wow Shawn thanks! This is a lot of info. I get a little bewildered with the lower level stuff, I tried my best and here are the results...

1. You're right, I didn't need to specify Mac address, so I updated the variables and call to begin() such that it uses the fields you suggested. I'll have to change some things in my full program to use the IPAddress datatype, but no big deal. This didn't help the speed, but it does make the IDE complain less about deprecated overloads. I was just copying the examples from the modbus library, so I'm happy to bring things up to date. Thanks!

2. I changed to if (client.available() > 0) in my loop. Again, this didn't help with the speed, but I do like to know I'm doing things in a more preferable way.

3. Updating the call to the write() function in modbus-tcp.cpp#L202 made a HUGE difference. I was getting an average of 150ms reply time in this test-code, but only about 2ms average after the change. Unfortunately I'm getting a [ Class "Client" has no member "writeFully" ] error when trying to implement the further correction you suggested... I think I get the intent here, but it seems that the call isn't looking in the right place. "Go to Definition" takes me to Ln 256 of QNEthernetFrame.cpp, but I don't see a writeFully() in that file.)
I'm a little nervous about trying this fix with just write() because of the potential problems you mentioned. But it does work with this single coil example, so I'll try it on my full program just for kicks.

Here's the current code, and the change to modbus-tcp.cpp is exactly as you suggested above with write():

C++:
#include <QNEthernet.h> // Most optimized Ethernet library for Teensy 4.1 Seems to have resolved memory leak in other libraries
#include <ArduinoModbus.h> // Dependency ArduinoRS485\RS485.cpp must have line 209 edited to use Serial5 and associated pins.
#include <EEPROM.h> // Enable access to EEPROM on Teensy board

using namespace qindesign::network;
EthernetServer EthServer(502);
ModbusTCPServer modbusTCPServer;

void updateOutput(); // Updates all output pins to high/low based on Modbus coil values

// Global variables
IPAddress ip{ 192, 168, 1, 50 }; // Default IP address
IPAddress subnet{ 255, 255, 255, 0 }; // Default Subnet mask
IPAddress gw{ 192, 168, 1, 1}; // Gateway
uint8_t jumperMode = 0; // Set during setup() to select boot mode. 1=Load defaults, 2=Ethernet, 3=Serial

void setup() {
  for (uint8_t i=2; i<5; i++) { // Set up input pins
    pinMode(i, INPUT_PULLUP); // Set all 3 jumper positions as inputs
  }
  pinMode(13, OUTPUT);

  delay(1); // Wait for inputs to stabilize. Stops false jumperMode=1 on cold start

  // Set mode based on jumper position
  if (digitalRead(2)==LOW) {
    jumperMode=1; // unused in this example
  } else if (digitalRead(3)==LOW) { // Ethernet Connection
    jumperMode=2;
  } else if (digitalRead(4)==LOW) { // Serial Connection
    jumperMode=3;
  }

  if (!jumperMode) { // No jumper installed
    while(1); // Halt program
  }

  if (jumperMode==2) { // Ethernet mode
    //qn::Ethernet.begin(mac, ip, subnet); // Create object
    Ethernet.begin(ip, subnet, gw);
    EthServer.begin(); // Start listening for client connections
    modbusTCPServer.begin(); // Start modbus service
    modbusTCPServer.configureCoils(0, 1);
  }

  if (jumperMode==3) { // Serial mode
    ModbusRTUServer.begin(1, 19200); // Start modbus service
    ModbusRTUServer.configureCoils(0, 1); // Coils for turning all shutters off, and individual control of each channel
  }
}

void loop() {
  if (jumperMode==2) { // Ethernet loop
    EthernetClient client = EthServer.available();
    if (client) {
      modbusTCPServer.accept(client);
      if (client.available()>0) { // Updated to Shawn's suggestion
        if (modbusTCPServer.poll()) { // poll() returns true when a request is present
          updateOutput();
        }
      }
    }
  }

  if (jumperMode==3) { // Serial loop
    if (ModbusRTUServer.poll()){
      updateOutput();
    }
  }
}

void updateOutput() {
  boolean state=(jumperMode==2)?modbusTCPServer.coilRead(0):ModbusRTUServer.coilRead(0);
  digitalWrite(13, state);
}
 
You’re right about writeFully(). I didn’t look at that Modbus library closely. They’re using “Client” and not “EthernetClient”. Ignore the writeFully() note for now. (But write() calls should still technically be called repeatedly until they’re done with the input. See that README link I posted for how. If you like.)

And yes, the first two notes won’t affect the speed.
 
... See that README link I posted for how. If you like...
Alright, new day! Yep I was looking at that, but couldn't quite figure out how to fit it into the code last night. Again, I'm not great with this low level stuff. But I tinkered with it this morning and I think I was able to get the same functionality cobbled into modbus-tcp.cpp#L202. Here's what I changed it to:

C++:
    // return ctx_tcp->client->write(req, req_length); // Original line 202
    size_t retval=0;
    while (req_length > 0 && ctx_tcp->client) {
        retval = ctx_tcp->client->write(req, req_length);
        req_length -= retval;
        req += retval;
    }
    ctx_tcp->client->flush();
    return retval;

This seems to work just as well as your original suggestion (no errors, and <3ms return time with my full program running 40+ registers and 20+ coils.)
I'm assuming the above code is doing the job of writeFully(), but I'm a little unsure of how I declared retval=0 in case the while() never happens. Does this look like the return will always be correct, or should I declare/order things differently?

I've actually played with several modbus libraries and they all seem to have some shortcomings, but this one seems to be simple to use and fairly up-to-date. I looked at Paul's fork, but it seems to be behind now even though it's probably more optimized for teensy. I wish there was a fork of ArduinoModbus that uses QNEthernet directly, but if this little fix doesn't break anything else, I'm happy to keep it in place.

Thanks again for your help!
 
Yeah, I expect it to be about the same speed. It’s just the proper way because write() may not send all the bytes requested of it. Since you’re using something like Modbus, I’m guessing that reliability is important to you.

Yes, it looks similar to what writeFully() does under the covers. Check out the library source code to confirm.

As for the library using QNEthernet, because I made EthernetClient conform to the Client interface included with Teensyduino, it shouldn’t matter too much. It’s just that that Modbus library could stand to be a little more robust when using Client::write() (actually Print::write()). It’s in the Arduino API docs that write() returns the number of bytes actually written; that’s why it needs to be checked.
 
Thanks Shawn,
That was my concern... Since write() may be called more than once in the loop but only returns once I couldn't quite gather if the way I have it is returning the correct value. It does seem to work, but yes indeed reliability is important, so I didn't want to jump to conclusions. I did do my best to duplicate the source of writeFully() in this new location.

I'll keep playing with it, but I think I'm up and running. Ethernet is running faster than Serial(9600) again. All is right in the world ;)
 
I just added an option to flush after all write calls. Other than the issue of write() not actually sending all the bytes (eg. because of internal buffers and such), just setting this new option in qnethernet_opts.h should duct tape the original issue (again, assuming the write() packets are small enough). Note that it might not matter for, say, the Modbus use case, it might affect TCP performance for other use cases. The new option: QNETHERNET_FLUSH_AFTER_WRITE
 
Back
Top