Teensy 4.1 Native Ethernet max SocketSize lower than set?

hijomojo

Member
Hi all,

I've been testing the ethernet capabilities of the Teensy 4.1 for receiving large UDP packets, and I've run up against a limit which I'm not learned enough to understand or solve.

I've got a minimal sketch which, after establishing the ethernet connection, just checks the size of the packet coming in and spits that out the USB serial.
At first I ran into the issue of packets being dropped entirely when they were larger than 2048 bytes, which I figured out was due to the SocketSize being set to precisely that. In testing of increasing this value, there seems to be a hard limit at 10212 bytes, even though I've seen samples floating around which have Ethernet.setSocketSize(1024 * 16); i.e. 16k, well larger than the 10 and a bit k I can actually achieve. I've pasted the arduino sketch below, and am using Max to send UDP packets of given sizes to test what comes back with "Received packet of size ". I've tried increasing StackHeap size, and reducing the SocketNum, but neither (and no combinations) seem to increase the acceptable max size of an incoming packet.

Does anyone have ideas why this might be happening, and whether it's possible to increase beyond this point? Or is there a hardware limitation of some sort at play here?
My ideal target for a project I'm working on is 36225 bytes, so there's a long gap to bridge!

Any help is much appreciated!

Code:
#include <NativeEthernet.h>
#include <NativeEthernetUdp.h>

// Any group of digital pins may be used
// Enter an IP address for your controller below.
// The IP address will be dependent on your local network:
IPAddress ip(192, 168, 2, 177);

unsigned int localPort = 3333;      // local port to listen on

// An EthernetUDP instance to let us send and receive packets over UDP
EthernetUDP Udp;

void setup() {
  Serial.begin(9600);
  uint8_t mac[6];
  teensyMAC(mac);
  Ethernet.setStackHeap(1024 * 64);
  Ethernet.setSocketSize(1024 * 16);
  Ethernet.setSocketNum(1);
  Ethernet.begin(mac, ip);
  Serial.println("Ethernet started");
  if (Ethernet.hardwareStatus() == EthernetNoHardware) {
    while (true) {
      Serial.println("Ethernet hardware not found?!");
      delay(1000);
    }
  }
  if (Ethernet.linkStatus() == LinkOFF) {
    while (true) {
      Serial.println("Waiting for Ethernet cable to be connected.");
      delay(1000);
    }
  }
  Serial.println("Ethernet OK");
  Serial.print("IP address: ");
  Serial.println(Ethernet.localIP());
  Udp.begin(localPort);
  Serial.println("Started UDP sevice");
}

void loop() {
  int packetSize = Udp.parsePacket();
  if(packetSize) {
    Serial.print("Received packet of size ");
    Serial.println(packetSize);
    while(Udp.available()){
      Udp.read();
    }
  }
}

void teensyMAC(uint8_t *mac) {

  static char teensyMac[23];
  
  #if defined(HW_OCOTP_MAC1) && defined(HW_OCOTP_MAC0)
    Serial.println("using HW_OCOTP_MAC* - see https://forum.pjrc.com/threads/57595-Serial-amp-MAC-Address-Teensy-4-0");
    for(uint8_t by=0; by<2; by++) mac[by]=(HW_OCOTP_MAC1 >> ((1-by)*8)) & 0xFF;
    for(uint8_t by=0; by<4; by++) mac[by+2]=(HW_OCOTP_MAC0 >> ((3-by)*8)) & 0xFF;

    #define MAC_OK

  #else
    
    mac[0] = 0x04;
    mac[1] = 0xE9;
    mac[2] = 0xE5;

    uint32_t SN=0;
    __disable_irq();
    
    #if defined(HAS_KINETIS_FLASH_FTFA) || defined(HAS_KINETIS_FLASH_FTFL)
      Serial.println("using FTFL_FSTAT_FTFA - vis teensyID.h - see https://github.com/sstaub/TeensyID/blob/master/TeensyID.h");
      
      FTFL_FSTAT = FTFL_FSTAT_RDCOLERR | FTFL_FSTAT_ACCERR | FTFL_FSTAT_FPVIOL;
      FTFL_FCCOB0 = 0x41;
      FTFL_FCCOB1 = 15;
      FTFL_FSTAT = FTFL_FSTAT_CCIF;
      while (!(FTFL_FSTAT & FTFL_FSTAT_CCIF)) ; // wait
      SN = *(uint32_t *)&FTFL_FCCOB7;

      #define MAC_OK
      
    #elif defined(HAS_KINETIS_FLASH_FTFE)
      Serial.println("using FTFL_FSTAT_FTFE - vis teensyID.h - see https://github.com/sstaub/TeensyID/blob/master/TeensyID.h");
      
      kinetis_hsrun_disable();
      FTFL_FSTAT = FTFL_FSTAT_RDCOLERR | FTFL_FSTAT_ACCERR | FTFL_FSTAT_FPVIOL;
      *(uint32_t *)&FTFL_FCCOB3 = 0x41070000;
      FTFL_FSTAT = FTFL_FSTAT_CCIF;
      while (!(FTFL_FSTAT & FTFL_FSTAT_CCIF)) ; // wait
      SN = *(uint32_t *)&FTFL_FCCOBB;
      kinetis_hsrun_enable();

      #define MAC_OK
      
    #endif
    
    __enable_irq();

    for(uint8_t by=0; by<3; by++) mac[by+3]=(SN >> ((2-by)*8)) & 0xFF;

  #endif

  #ifdef MAC_OK
    sprintf(teensyMac, "MAC: %02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
    Serial.println(teensyMac);
  #else
    Serial.println("ERROR: could not get MAC");
  #endif
}
 
This is more likely to be related to the MTU (maximum transfer unit) set on your network switch or receiving computer. I think there may also be some MTU set in either NativeEthernet.h or FNET.h that may also prevent you from getting packets larger than ~1400bytes. I would very much like to be proven wrong though, it's been a headache for me as well.
 
I'm curious - why do you care about jumbo frames vs just sending smaller ones and combining them? Only substantial drawback I know of is when there is packet loss.

I wouldn't count on other equipment supporting more than 9216 bytes.
 
This is more likely to be related to the MTU (maximum transfer unit) set on your network switch or receiving computer.
Yeah, that was actually one of the first things I thought after some research. I can't change the MTU in the router I'm currently testing with (typical ISP-provided junk), so I decided to just test it and see what happens. In Max I can send and receive up to a 32768 byte packet through the router without issue (via multicast to force it out through the router instead of it looping back internally). So that leads me to believe MTU isn't having any impact. One thing I did read is that large packets like this can just be silently sliced up and reassembled by hardware in between, so perhaps that's happening and I just don't see any side effect on the receiving end.

I think there may also be some MTU set in either NativeEthernet.h or FNET.h that may also prevent you from getting packets larger than ~1400bytes. I would very much like to be proven wrong though, it's been a headache for me as well.
Hmm, interesting point. I might have another poke around in there. These kinds of technicalities in networking and the resulting complexity of the libraries is a bit beyond me though, unfortunately.

why do you care about jumbo frames vs just sending smaller ones and combining them?
At the moment I'm aiming to send as many (video) frames as possible via this method; 12075 pixels x 3 colours = 36225 bytes, and I'm targeting at least 15fps, but ideally as close as I can get to 60 which is roughly the limit of the Teensy 4.1 for my output configuration via OctoWS. On the Teensy side, does receiving more but smaller frames result in any performance loss compared to fewer but larger frames? In theory 60fps at this B/frame is 2.1735 MB/s, or 17.3Mbit/s, which is well beneath the 100Mbit the Teensy 4.1 ethernet is apparently rated to.

At the moment on the Max side, the process of splitting the video frames into numerous smaller packets slows things down immensely...from 60+ fps down to about 12 when sending as 2048 byte sections, this this is currently a big factor for me. Though...in writing this just now, I thought of a potential alternative method which might help me here.

I wouldn't count on other equipment supporting more than 9216 bytes.
Thanks for the tip. Any reason behind that specific value? 9KiB seems a strange value to me. Hopefully I should be okay, as the network will be very simple, just 2 (or 4, if I really need to) Teensy 4.1s connected directly to the computer via an ethernet hub.
 
Last edited:
Okay, that's good to hear. At the moment the speed limitations I'm getting (in effective fps) are definitely on the Max side.

I've just been working on that idea I alluded to in my previous reply and have found that, even in this first attempt, I can get around 40 frames per second worth of data sent to and confirmed by USB serial response (slow!) from the Teensy. I achieved this by using a different method for splitting each video frame into subsections in Max (I won't get off topic explaining this, suffice to say some Max objects out-perform others for the same purposes). At the moment this is with packets of 525 pixels == 1575 bytes + header, so just into Jumbo Frame territory. I didn't realise what I was doing before was technically into Super Jumbo Frame territory. I'll continue experimenting with different packet sizes by splitting the video frames different ways to see what's most efficient.

This round of testing was also validated on what is intended to be the actual network topology (Win10 > Eth Hub > Teensy), too, which is a good sign.

All this is great, but I guess the original question still stands regarding the peculiar 10212 byte limit (which exists in my original router-based network topology and the hub-based topology, so seems to be in the Teensy). It looks like it's not going to affect my situation now, but perhaps someone in the future will run into this in a project where it might not be so easily avoided.
 
FNET supports ip fragmentation so the packets are already being split up and reassembled automatically, with ip fragmentation jumbo frames are not being sent (nor does FNET support them as far as I'm aware) so you are free to use more than the maximum MTU size without difficulties. I know FNET does support more than the 10212 byte limit you are hitting, I've done tests with 32kB+ sockets (though this wasn't through the use of the NativeEthernet library from what I remember) so I'm curious where that weird limit is coming into play.

As far as having enough throughput for what you are trying to do there should be no problems with it, manitou has done speed tests of a few different ethernet solutions here: https://forum.pjrc.com/threads/60532-Teensy-4-1-Beta-Test?p=237096&viewfull=1#post237096
Quick summary below (red highlight is NativeEthernet library):
Code:
                         Ethernet performance
                   T41e   1062SDK  [COLOR="#FF0000"]T41fnet[/COLOR] T41USBe  T35e  T4+W5500    info
TCP xmit (mbs)       73        87     [COLOR="#FF0000"]92[/COLOR]       78     59         9 
TCP recv (mbs)       93        71     [COLOR="#FF0000"]78[/COLOR]       30     81        11  

UDP xmit (mbs)       97        97     [COLOR="#FF0000"]97[/COLOR]       95     85        11   blast 20 1000-byte pkts
UDP xmit (pps)   149476    137453 [COLOR="#FF0000"]152052[/COLOR]    32331  66534     21514   blast 1000 8-byte pkts
UDP recv (mbs)       91        95     [COLOR="#FF0000"]96[/COLOR]       40     67         9   no-loss recv of 20 1000-byte pkts
UDP RTT (us)         94       104    [COLOR="#FF0000"]104[/COLOR]     1651    183       150   RTT latency of 8-byte pkts

ping RTT (us)       120       108    [COLOR="#FF0000"]103[/COLOR]     2000    127        82 

ePower (ma)          59       100     [COLOR="#FF0000"]59[/COLOR]      174    100       132   ethernet module current
 
  tests on 100mbs full-duplex Ether with linux box on switch
  T41fnet and T41USBe  FNET TCP/IP native (arduino wrapper) and USB host, no threads
  W5500 SPI @37.5MHz, 2KB buffers
 
Thanks very much for the reply, vjmuzik, and especially thanks for your work on these libraries.
Thank you also for the assurance of the throughput capabilities, along with the numbers and link to the other thread.

Do you happen to have any ideas, at the moment, where the 10212 byte limit might be coming from? I've had to take a few days off working on the project this relates to at the moment, but I'm trying to think of some tests I can to do, once I get back into it, which might help identify the cause. At the moment I'm thinking to try sending packets from another program like Processing rather than Max, just in case there's something funny going on with Max (even though it can receive its own 32k frames looping through the router fine). I do have a rather tight time limit I'm working within unfortunately, so for the time being have to limit how much R&D and learning I can do to things which are realistically within reach of my current skillset. Thought, if I/we don't figure out what might be happening for a while, and you think it's worth looking into further, I'm happy to help out with what I can once I have more than a glimpse of spare time :)
 
Formatting JSON HTML Posts

Hello All,
Having trouble finding the right thread for this question.....
I'm trying to modify the WebClient example for Teensy 4.1. The google connection worked...this gave me enough joy to jump in way over my head.
Moving on a bit, I'm having an enormously difficult time finding some references on making json POSTs.
My code is really, a mess at this point, and I don't have any hopes to build from it without learning more about the c++ format for json POSTs, but here it is for the morbidly curious. :)
Thanking anyone in advance for some references from which to study this.
-John

Code:
/*  Web client  
This sketch connects to a website ([url]http://www.google.com[/url])
 using an Arduino Wiznet Ethernet shield.

 Circuit: Teensy 4.1, native ethernet.

 created 18 Dec 2009
 by David A. Mellis
 modified 9 Apr 2012
 by Tom Igoe, based on work by Adrian McEwen
 Then hacked beyond recognition by yours truly.*/

//#include <SPI.h>
//#include <ArduinoHttpClient.h>
//#include <Ethernet.h>
#include <Bridge.h>
#include <NativeEthernet.h> // For internal T41 ethernet.
#define PIN_13_TEENSY_LED (13)

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
//IPAddress server(74,125,232,128);  // numeric IP for Google (no DNS)

char server[] = "www.google.com";    // name address for Google (using DNS)
char authQuery[] = "\"sn\": \"1234567890\",\"accesskey\": \"F43cab98-8cd8-8349-8a9b-c2fabcd9fdb4\"";

// Set the static IP address to use if the DHCP fails to assign
//IPAddress ip(192, 168, 10, 176);  //was 192:168:0:177
IPAddress ip(192, 168, 137, 177);
IPAddress myDns(192, 168, 137, 1);

// Initialize the Ethernet client library with the IP address and port of the server that you want to connect to (port 80 is default for HTTP):
EthernetClient client;


// Variables to measure the speed
unsigned long beginMicros, endMicros;
unsigned long byteCount = 0;
bool printWebData = true;  // set to false for better speed measurement
int TimeIt;

void setup() {  // You can use Ethernet.init(pin) to configure the CS pin
  pinMode(PIN_13_TEENSY_LED, OUTPUT);
  Serial.begin(115200);
  while (!Serial){ ; }  //wait for port

  // start the Ethernet connection:
  Serial.println("Try DHCP:");
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // Check for Ethernet hardware present
    if (Ethernet.hardwareStatus() == EthernetNoHardware) {
      Serial.println("Ethernet shield was not found.  Sorry, can't run without hardware. :(");
      while (true) {        delay(1); }// do nothing, no point running without Ethernet hardware
    }
    if (Ethernet.linkStatus() == LinkOFF) {      Serial.println(" ");    Serial.println("Ethernet cable is not connected.");   Serial.println(" "); }
    // try to congifure using IP address instead of DHCP:
    Serial.print("Ethernet.begin(mac, ip, myDns)");
    Ethernet.begin(mac, ip, myDns);        Serial.println(" .....Done.");
  
  } else {    Serial.print("  DHCP assigned IP ");    Serial.println(Ethernet.localIP());
  }

  Serial.print("Calling getIpAddress()... ");  getIpAddress();    Serial.println(" ...getIpAddress() done.");

  // give the Ethernet shield a second to initialize: 
  delay(1000);

 /* This is the part that hangs up...... */
 Serial.println("Writing POST strings");
client.println("POST 'http://dev-gateway.trafficloud.io/v1/authd'");
client.println("'Content-Type: application/json'");
client.println("'{");
client.println(authQuery);
client.println("}'");
Serial.println("POST string written");
while (client.available() > 0) {      char c = client.read();      Serial.print(c);    }
 
  Serial.println(" ");  Serial.println("Try tbcs.connect"); Serial.println(" ");
//    if (client.connect(tbcsAuth, 80)){
//        Serial.print("connected to ");    Serial.println(client.remoteIP());
        //client.stop();
      /*
      Bridge.begin(tbcsAuth);
      Serial.print("Bridge.begin() OK ???");
    
      Process p;    // Create a process and call it "p"  // Curl -X POST -H "Content-Type: application/json" -d '{"username":"abc","password":"abc"}'         [url]https://api.example.com/v2/login[/url]
      p.begin("curl"); 
      p.addParameter("-X POST");
      p.addParameter("-H "); 
      p.addParameter("Content-Type: application/json");
      p.addParameter("-d");
      p.addParameter("http://dev-gateway.trafficloud.io/v1/authd");
      p.run();    // Run the process and wait for its termination
      Serial.println("Curl POST finished... ");
      while (p.available() > 0) {      char c = p.read();      Serial.print(c);    }
      */
//    }else{
//      Serial.println("*********");
//      Serial.println("NO CONNECT TO GATEWAY");
//      Serial.println("*********");
//    }
    //if (!client_tbcs.connected()){  Serial.println("STOPPING client_tbcs");  client_tbcs.stop();  }

  
  Serial.print("connecting to ");  Serial.print(server);  Serial.println("...");
  // Bridge.begin(server);
  
  // if you get a connection, report back via serial:
  if (client.connect(server, 80)) {
    Serial.print("connected to ");
    Serial.println(client.remoteIP());
    // Make a HTTP request:
      /*
    client.println("GET /search?q=arduino HTTP/1.1");
    client.println("Host: www.google.com");*/
    
    client.println("POST dev-gateway.trafficloud.io/v1/authd); // /post HTTP/1.1");
    client.println("Content-Type: application/json");
    client.println("Content-Length: 1024");
    // client.println("Host: dev-gateway.trafficloud.io/v1/authd");
    
    client.println("Connection: close");
    client.println();
  } else {     // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
  beginMicros = micros();


  
}

void loop() {
  // if there are incoming bytes available
  // from the server, read them and print them:

  int len = client.available();
  if (len > 0) {
    byte buffer[80];
    if (len > 80) len = 80;
    client.read(buffer, len);
    if (printWebData) {
      Serial.write(buffer, len); // show in the serial monitor (slows some boards)
    }
    byteCount = byteCount + len;
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    endMicros = micros();
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    Serial.print("Received ");
    Serial.print(byteCount);
    Serial.print(" bytes in ");
    float seconds = (float)(endMicros - beginMicros) / 1000000.0;
    Serial.print(seconds, 4);
    float rate = (float)byteCount / seconds / 1000.0;
    Serial.print(", rate = ");
    Serial.print(rate);
    Serial.print(" kbytes/second");
    Serial.println();

    // do nothing forevermore:
    while (true) {
      digitalWrite(PIN_13_TEENSY_LED, HIGH);    delay(50);
      digitalWrite(PIN_13_TEENSY_LED, LOW);     delay(50); 
    }
  }
}


void getIpAddress() 
{
  // start the Ethernet connection:
  String addr;
  Serial.println("=== Waiting for IP address");

  teensyMAC(mac); // Get MAC address from Teensy
  
  TimeIt = millis();
  while (Ethernet.begin(mac) == 0) 
  {
    if(millis() - TimeIt > 10000)  // Ten seconds to get a DHCP served address
    {
      Serial.println("Failed to configure Ethernet using DHCP");
      getFixedIpAddress();
      break;
    }
  }

  addr=String(Ethernet.localIP()[0]) + "." +
       String(Ethernet.localIP()[1]) + "." +
       String(Ethernet.localIP()[2]) + "." +
       String(Ethernet.localIP()[3]);

  Serial.print("IP address is: "); Serial.println(Ethernet.localIP());
}

/*********************** getFixedIpAddress *********************/
void getFixedIpAddress() 
{
  //IPAddress ip(169, 254, 5, 25);
  //IPAddress ip(192, 168, 137, 176);
  
  // start the Ethernet connection:
  Serial.println("=== Setting fixed IP address");
  
  Ethernet.begin(mac, ip);

} // end getFixedIpAddress

/*********************** teensyMAC *********************/
void teensyMAC(uint8_t *mac) 
{
    for(uint8_t by=0; by<2; by++) mac[by]=(HW_OCOTP_MAC1 >> ((1-by)*8)) & 0xFF;
    for(uint8_t by=0; by<4; by++) mac[by+2]=(HW_OCOTP_MAC0 >> ((3-by)*8)) & 0xFF;
    Serial.printf("MAC: %02x:%02x:%02x:%02x:%02x:%02x\n", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
}
 
Last edited:
Moving on a bit, I'm having an enormously difficult time finding some references on making json POSTs.
My code is really, a mess at this point, and I don't have any hopes to build from it without learning more about the c++ format for json POSTs, but here it is for the morbidly curious. :)


HTTP requires "\r\n" as a line ending for every header line and for the mandatory blank line before the content body.

So change lines like:
Code:
client.println("POST 'http://dev-gateway.trafficloud.io/v1/authd'");
client.println("'Content-Type: application/json'");
client.println("'{");
client.println(authQuery);
client.println("}'");
to more like:
Code:
client.printf ("%s\r\n", "POST http://dev-gateway.trafficloud.io/v1/authd HTTP/1.1");
client.printf ("%s\r\n", "Content-Type: application/json");
client.printf ("\r\n");
client.printf ("%s\r\n", "'{");
client.printf ("%s\r\n", authQuery);
client.printf ("%s\r\n", "}'");

HTTP format is rigid, you have to follow the rules as given in the relevant RFCs. You had missed the HTTP/1.1 protocol tag in the POST line, as well as added spurious single quotes at various points - don't try to guess these things, its easiest to snoop on the trafic of a working example, for instance using browser debug facilities.


There may be a good library to do this for you though - look around.
 
Having trouble finding the right thread for this question.....

It's best you create one specifically for this question, no? This is completely off topic from what I can see....

If a moderator agrees, would you mind please moving amensch's post and MarkT's answer to their own thread?
 
Cheers for coming back to this old(ish) thread to link me that!
Unfortunately I had to jump off the project last year (covid cancelation) however it's something I'm looking to finish when I have some time in the coming months. I did just have a read through of that thread and it indeed sounds promising. Will definitely be back here to update with any developments/feedback/further thanks.
I appreciate the help!
 
A bit too late, but my two cents.

This is more likely to be related to the MTU (maximum transfer unit) set on your network switch or receiving computer. I think there may also be some MTU set in either NativeEthernet.h or FNET.h that may also prevent you from getting packets larger than ~1400bytes. I would very much like to be proven wrong though, it's been a headache for me as well.

Just a (maybe too long) explanation:

MTU is only limiting when the ‘dont-fragment’ flag in an ip packet is set. Otherwise the networking stack should break the packet up in several packets taking into account the MTU. Default is about 1500. For a lot of unmanaged switches and routers it is not possible to change the MTU, but they are still capable of sending packets larger than the MTU (as long as the df flag is not set). So I don’t think the MTU comes into play.

You can try this out easily with not necessarily a teensy, but your laptop or desktop. Ping on Linux and Windows do have a dont-fragment flag: first ping a target with a packetsize of 1600. This is to test whether a target replies ping requests of this size. Then retry pinging with the df flag. When the MTU is set around 1500, you should not get a reply. Most of the time the limit is 1472 bytes when the MTU is 1500, but some network technologies can limit this (vxlan, vpn, etc).

I’ve never seen MTU > 9216. The Cisco 93180YC-FX is quite a beast, but the max. MTU for that switch is 9216 bytes.

Sorry for giving too much detail, but I don’t want you to go on a wild goose chase when it comes to the MTU.

With regards to the real question: I don’t know. But I hope it sheds a little light on the MTU.
 
Back
Top