I don't own a Mega and no shield for it, sorry.
Could you run the second code on a a second T4.1? I only used a Mega2560 as one was available.
I don't own a Mega and no shield for it, sorry.
Could you run the second code on a a second T4.1? I only used a Mega2560 as one was available.
@UdoZ - Could you translate your code's comments to English? Sadly, I don't speak German. I tried copying a few of the comments into Google Translate....
My understanding is I'm supposed to run this test using Teensy 4.1 (running the 1st program) connected to an Arduino Mega (running the 2nd program) with Arduino Ethernet Shield. Is that right? Can you tell me exactly which Ethernet Shield you used? Subtle differences exist between the W5100, W5200, W5500 chips used on the many ethernet shields. Did you connect the 2 boards directly with a cable, or were they both plugged into an ethernet switch or hub?
And I know this question may be redundant, but please tell me you actually ran this EXACT code on those 2 boards and saw the problem, right? Many, many times on this forum we've seen a program like this which is adapted from much larger code. Sometimes people do not actually test the reduced code. I'm not trying to say you would do such a thing, only that many times I've put hours of work into running a program like this and ultimately it was unable to reproduce the problem because it lacked some detail from the larger original code.
Please, confirm you actually ran these exact 2 programs, and tell me exactly which non-Teensy shield was used with Arduino Mega.
Hi Paul, yes, these exactly two programs have been used.
The problem is infrequent, and one need patience to see it. Its low frequency explains, why the fault is less frequently seen on applications with low Ethernet traffic.
Confirmed, the Teensy 4.1 stopped printing after about 40 minutes. The Arduino Mega side started printing "connection # ... failed" messages.
Later today I will work on reproducing the error more rapidly....
Sorry for the my silence on these issues recently, but I haven't had time to dedicate towards this. I'm aware of what's happening with the TCP code and why it's seemingly running out of sockets out of nowhere. This was previously reported here: https://github.com/vjmuzik/NativeEthernet/issues/7
The short of it is that when EthernetServer.available finds a client it opens a socket and expects the client to send data first before returning with an active client. From what I've been able to deduce with Wireshark is that FNET is losing that TCP data seemingly at random intervals meaning that EthernetServer.available will never close the socket because it's still expecting data to arrive. Eventually you will run out of sockets, see the link above to easily reproduce this without having to wait 40 minutes.
Unfortunately aside from implementing a blocking timeout on EthernetServer.available to detect the socket that doesn't receive data I haven't been able to pinpoint where exactly in FNET the data is getting lost. This is obviously a quick solution, but it was not one I was particularly happy with, so I never committed it, because it drops the socket altogether and I wanted to find the cause of the missing data.
// Run this on Teensy 4.1
//
// https://forum.pjrc.com/threads/67612?p=282662&viewfull=1#post282662
#define useClientClose 1
#include <Wire.h>
#include <SPI.h>
#include <NativeEthernet.h>
#include <elapsedMillis.h>
#include <TimeLib.h>
byte mac[] = {0xA8, 0x61, 0x0A, 0xAE, 0x2A, 0x1B };
IPAddress ip(192, 168, 2, 11);
IPAddress gateway(192, 168, 2, 1);
IPAddress subnet(255, 255, 255, 0);
EthernetServer server(9010);
float Parameter[275];
int LEDred = 13;
int LEDyellow = 14;
byte OutBuffer[45];
const byte maxPowerMeasurements = 10;
float fPges[1441][maxPowerMeasurements];
unsigned int dayCounter[maxPowerMeasurements] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
float Estunde[24][maxPowerMeasurements]; //hier speichern wir die stündlichen Summen ab
int counterEstunde[maxPowerMeasurements] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
int counterEtag[maxPowerMeasurements] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
float Etag[maxPowerMeasurements]; //hier speichern wir die laufende Tagessumme ab, dh Summe aller Messwerte durch Anzahl der Messwert
float oldEtag[maxPowerMeasurements]; //hier den gestrigen Wert, so dass wir ihn nach Miternacht für 24 Stunden abrufen können
byte letzteStunde = 26;
float LastPowerMeasurement[maxPowerMeasurements];
int EndSwitches = 10;
boolean useClientClose_PC_Contact = true;
void setup()
{
Serial.begin(250000);
while (!Serial) ; // wait for serial monitor
Serial.println("NativeEthernet Crash Test");
if (CrashReport) {
Serial.print(CrashReport);
Serial.println("send any text to start again");
while (!Serial.available()) ; // wait
Serial.clear();
}
//while (1);
//Ignore these inits, they are caused by my snippet
OutBuffer[ 0] = 0x47; OutBuffer[ 1] = 0x45; OutBuffer[ 2] = 0x54; OutBuffer[ 3] = 0x20; OutBuffer[ 4] = 0x2F; OutBuffer[ 5] = 0x63; OutBuffer[ 6] = 0x6F; OutBuffer[ 7] = 0x6E; OutBuffer[ 8] = 0x74; OutBuffer[ 9] = 0x72; //GET /contr
OutBuffer[10] = 0x6F; OutBuffer[11] = 0x6C; OutBuffer[12] = 0x2E; OutBuffer[13] = 0x68; OutBuffer[14] = 0x74; OutBuffer[15] = 0x6D; OutBuffer[16] = 0x6C; OutBuffer[17] = 0x3F; OutBuffer[18] = 0x70; OutBuffer[19] = 0x6F; //ol.html?po
OutBuffer[20] = 0x77; OutBuffer[21] = 0x65; OutBuffer[22] = 0x72; OutBuffer[23] = 0x3D;
Parameter[0] = 1;
pinMode(LEDred, OUTPUT);
pinMode(LEDyellow, OUTPUT);
Serial.println(F("connecting to Ethernet"));
pinMode(4, OUTPUT); digitalWrite(4, HIGH);
Ethernet.begin(mac, ip, gateway, subnet);
//server.begin(); //start listening for clients on EThernet
delay(1000);
}
void loop()
{
PCContact();
delay(1);
}
//--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
void PCContact()
{
char ReadText[120];
byte Count = 0;
byte Action = 99;
long t1 = millis();
Ethernet.setRetransmissionCount(1); //<-------------------------------------------------------------------------------------------------------------------
Ethernet.setRetransmissionTimeout(50); //<-------------------------------------------------------------------------------------------------------------------
EthernetClient client = server.available();
client.setConnectionTimeout(25);
boolean ExtraLine = false;
if (client) {
//wait for Data from client
elapsedMillis waitPC1;
//wait for Server to connect
while (!client.available() && waitPC1 < 10000) {
delayMicroseconds(1);
if (waitPC1 > 16 && waitPC1 % 16 == 0) checkSerial1_for_KNX();
}
if (waitPC1 > 1000) {
Serial.print("waitPC1: "); //INFO ONLY
Serial.print(waitPC1);
Serial.print("\tBytes:");
Serial.println(client.available());
}
if (client.available()) {
Serial.println();
while (client.available() && Count < 95) {
ReadText[Count] = client.read();
if (ReadText[Count] > 48) Serial.print(ReadText[Count]);
Count++;
}
if (ReadText[0] == 'G' && ReadText[1] == 'E' && ReadText[2] == 'T') {
Action = AnalyseGet(ReadText); //Decode Get Information
boolean currentLineIsBlank = true;
int Limiter = 0;
//In case we haven't already read everything
while (client.available() && Limiter < 120) {
char c = client.read(); Serial.print(c); // see what has been received
if (c == '\n') {
currentLineIsBlank = true; // you're starting a new line
} else if (c != '\r') {
currentLineIsBlank = false; // you've gotten a character on the current line
}
if (c == '\n' && currentLineIsBlank) break;
Limiter++;
}
//Response - usually this connection goes to a true HTTP Server
client.println(F("HTTp/1.1 200 OK")); client.println(F("Content-Type: text/html")); client.println(F("Connnection: close")); client.println();
Serial.print("\tAction:"); Serial.print(Action); byte SockNr = client.getSocketNumber(); Serial.print("\tSkNr@Start:"); Serial.print(SockNr);
switch (Action) {
//-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
case 0: //Info not understood
{
Serial.print(F("\n\n\tCannot decode PC's GET Request. RXed: ")); Serial.print(ReadText); Serial.print(F(" Count=")); Serial.println(Count);
client.println(F("NOK"));
break;
}
//-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
case 99: //favicon - nothing to do
{
Serial.println(F("Favicon"));
break;
}
//====================================================================================================================================================================
case 20: //sende Pges Daten an den PC
{
//There is nothing special about this case 20, it has been stripped from my major application for this test - it basically sends
//in this test example : 0$0.00$0.00$0.00$0.00$0$27$53$1$1$1970$0$0.00$0$0.00$0.00$DATA$DATA
// else if(ReadText[5]=='P' && ReadText[6]=='g' && ReadText[7]=='E' && ReadText[8]=='s' && ReadText[9]=='N') bReturn=20; //Messwerte der letzten Zeit senden
//GET /PgEsNpq_FabcdTvxyz$VAB$
//012345678901234567890123
if (ReadText[10] != ReadText[11] || ReadText[10] < 48 || ReadText[10] > 57) {
Serial.print(F("Invalid Request: "));
Serial.println(ReadText);
break;
}
int PNr = ReadText[10] - 48;
int StartIndex = 0; int EndIndex = dayCounter[PNr];
//bei partieller Anfrage gilt GET /PgEsNpq_FabcdTvxyz$VAB$
// 012345678901234567890123
// p=q ParameterNr
// FROM
// To .it VAB=V-1 für gestern
if (PNr == 9) ExtraLine = true;
if (ReadText[13] == 'F' && ReadText[18] == 'T' && ReadText[23] == '$') {
int S = 0;
int E = dayCounter[PNr];
//partielle Anfrage
if ( (ReadText[14] == '0' || ReadText[14] == '1') && ReadText[15] >= 48 && ReadText[15] <= 57 && ReadText[16] >= 48 && ReadText[16] <= 57 && ReadText[17] >= 48 && ReadText[17] <= 57) {
S = (ReadText[14] - 48) * 1000 + (ReadText[15] - 48) * 100 + (ReadText[16] - 48) * 10 + (ReadText[17] - 48);
} else {
//Serial.print(F("14 15 16 17: ")); Serial.print(ReadText[14]); Serial.print(ReadText[15]); Serial.print(ReadText[16]); Serial.println(ReadText[17]);
}
if (ReadText[19] != 'x' || ReadText[20] != 'x' || ReadText[21] != 'x' || ReadText[22] != 'x' ) {
if ( (ReadText[19] == '0' || ReadText[19] == '1') && ReadText[20] >= 48 && ReadText[20] <= 57 && ReadText[21] >= 48 && ReadText[21] <= 57 && ReadText[22] >= 48 && ReadText[22] <= 57 ) {
E = (ReadText[19] - 48) * 1000 + (ReadText[20] - 48) * 100 + (ReadText[21] - 48) * 10 + (ReadText[22] - 48);
}
} else {
//Serial.print(F("19 20 21 22: ")); Serial.print(ReadText[19]); Serial.print(ReadText[20]); Serial.print(ReadText[21]); Serial.println(ReadText[22]);
}
if (E >= S) {
StartIndex = S; EndIndex = E;
} else {
//Serial.print(F("E: ")); Serial.print(E); Serial.print(F(" S: ")); Serial.println(S);
}
} else {
//Serial.print(F("13 18 23: ")); Serial.print(ReadText[13]); Serial.print(ReadText[18]); Serial.println(ReadText[23]);
}
//Serial.print(F("Pges[")); Serial.print(PNr); Serial.print(F("] -> PC from: ")); Serial.print(StartIndex); Serial.print(F(" to ")); Serial.print(EndIndex);
if (ReadText[24] == 'V' && ReadText[25] == '-' && ReadText[26] == '1' ) {
client.print(dayCounter[PNr]); client.print(F("$"));
client.print(Parameter[267]); client.print(F("$"));
client.print(Parameter[268]); client.print(F("$"));
client.print(Parameter[269]); client.print(F("$"));
client.print(LastPowerMeasurement[PNr]); client.print(F("$"));
client.print(hour()); client.print(F("$"));
client.print(minute()); client.print(F("$"));
client.print(second()); client.print(F("$"));
client.print(day()); client.print(F("$"));
client.print(month()); client.print(F("$"));
client.print(year()); client.print(F("$"));
client.print(counterEstunde[PNr]); client.print(F("$"));
client.print(Estunde[letzteStunde][PNr]); client.print(F("$"));
client.print(counterEtag[PNr]); client.print(F("$"));
client.print(Etag[PNr]); client.print(F("$"));
client.print(Parameter[EndSwitches + 1]);
client.print(F("$DATA$"));
for (int i = StartIndex; i < EndIndex; i++) {
client.print(F("$"));
if (i % 100 == 0) checkSerial1_for_KNX();
}
client.print(F("DATA"));
}
else
{
client.print(dayCounter[PNr]); client.print(F("$"));
client.print(Parameter[267]); client.print(F("$"));
client.print(Parameter[268]); client.print(F("$"));
client.print(Parameter[269]); client.print(F("$"));
client.print(LastPowerMeasurement[PNr]); client.print(F("$"));
client.print(hour()); client.print(F("$"));
client.print(minute()); client.print(F("$"));
client.print(second()); client.print(F("$"));
client.print(day()); client.print(F("$"));
client.print(month()); client.print(F("$"));
client.print(year()); client.print(F("$"));
client.print(counterEstunde[PNr]); client.print(F("$"));
client.print(Estunde[letzteStunde][PNr]); client.print(F("$"));
client.print(counterEtag[PNr]); client.print(F("$"));
client.print(Etag[PNr]); client.print(F("$"));
client.print(Parameter[EndSwitches + 1]); client.print(F("$DATA$"));
for (int i = StartIndex; i < EndIndex; i++) {
client.print(fPges[i][PNr], 4);
client.print(F("$"));
if (i % 100 == 0) checkSerial1_for_KNX();
}
client.print(F("DATA"));
}
break;
}
//-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
default:
{
break;
}
} //end e switch(Action)
ReadText[0] = 0;
} //ENDE else if(ReadText[0]=='G' && ReadText[1]=='E' && ReadText[2]=='T')
} //end if(client.available())
Serial.print(F("\t\tadditional Info rxed: "));
int addC = 0;
while (client.available()) {
char c = client.read(); Serial.print("0x"); Serial.print(c, HEX); Serial.print(" ");
addC++;
}
if (addC > 2) {
Serial.print(" = Nr of Bytes:");
Serial.print(addC);
} else {
Serial.print(F(" = None"));
}
#if defined (useClientClose)
if ( useClientClose_PC_Contact)
{
client.close(); Serial.print("\tclient.close done");
byte SockNr = client.getSocketNumber(); Serial.print(" SNrP1:"); Serial.print(SockNr);
}
#endif
client.stop(); Serial.print("\tclient.stop executed"); byte SockNr = client.getSocketNumber(); Serial.print(" SkNr@End:"); Serial.print(SockNr);
Serial.print(F("\tdt_ms: ")); Serial.println(millis() - t1);
if (ExtraLine) Serial.println();
} //end if(client)
} //End PCContact
//----------------------------------------------------------------------------------------------------------------------------
byte AnalyseGet(char ReadText[])
{
byte bReturn = 0;
if (ReadText[3] == ' ' && ReadText[4] == '/') {
if (ReadText[5] == 'f' && ReadText[6] == 'a' && ReadText[7] == 'v' && ReadText[8] == 'i' && ReadText[9] == 'c' && ReadText[10] == 'o' ) {
bReturn = 99; //Favicon ignore
} else if (ReadText[5] == 'P' && ReadText[6] == 'g' && ReadText[7] == 'E' && ReadText[8] == 's' && ReadText[9] == 'N') {
bReturn = 20; //Messwerte der letzten Zeit senden
}
}
return bReturn;
}
//============================================================================================================================================================================================================================
void checkSerial1_for_KNX()
{
//this routine checks 9600baud KNX Bus on Serial1
}
// Run this on Arduino Mega
//
// https://forum.pjrc.com/threads/67612?p=282662&viewfull=1#post282662
#include <Wire.h>
#include <SPI.h>
#include <Ethernet.h>
byte mac[] = {0xA8, 0x61, 0x0A, 0xAE, 0x2A, 0x10};
IPAddress ip(192, 168, 2, 185);
IPAddress myDns(192, 168, 2, 1);
EthernetClient client;
IPAddress server(192, 168, 2, 11);
unsigned long lastConnectionTime = 0; // last time you connected to the server, in milliseconds
const unsigned long postingInterval = 100; // delay between updates, in milliseconds
int i = 1;
void setup()
{
Ethernet.init(10); // Most Arduino shields
Serial.begin(115200);
Ethernet.begin(mac, ip, myDns);
Serial.print("My IP address: ");
Serial.println(Ethernet.localIP());
Ethernet.setRetransmissionCount(2); //<-------------------------------------------------------------------------------------------------------------------
Ethernet.setRetransmissionTimeout(50); //<-------------------------------------------------------------------------------------------------------------------
delay(1000);
}
void loop()
{
if (millis() - lastConnectionTime > postingInterval) {
for (int i = 1; i < 10; i++) {
Request(i);
delay(10);
}
}
}
// --------------------------------------
void Request(int i)
{
unsigned long t1 = millis();
if (client.connect(server, 9010)) {
Serial.print("\nconnecting..."); Serial.print(i);
switch (i) {
case 1: client.println("GET /PgEsN11"); break;
case 2: client.println("GET /PgEsN22"); break;
case 3: client.println("GET /PgEsN33"); break;
case 4: client.println("GET /PgEsN44"); break;
case 5: client.println("GET /PgEsN55"); break;
case 6: client.println("GET /PgEsN66"); break;
case 7: client.println("GET /PgEsN77"); break;
case 8: client.println("GET /PgEsN88"); break;
case 9: client.println("GET /PgEsN99"); break;
}
client.println();
byte Wait = 0;
while (client.available() < 10 && Wait < 25) {
Wait++;
delay(1);
}
Serial.print("\tWait:"); Serial.print(Wait); Serial.print(" "); Serial.print("got so far: "); Serial.print(client.available()); Serial.print(" ");
if (client.available() == 0) {
Wait = 0;
while (client.available() < 10 && Wait < 25) {
Wait++;
delay(1);
}
Serial.print("\textra wait:"); Serial.print(Wait); Serial.print(" "); Serial.print("after 25ms: "); Serial.print(client.available());
}
while (client.available()) {
char c = client.read(); Serial.write(c);
}
Serial.print("\tdt_ms:"); Serial.println(millis() - t1);
client.stop();
lastConnectionTime = millis();
} else {
Serial.println("connection #"); Serial.print(i); Serial.println(" failed");
}
}
I've been looking into this NativeEthernet bug. Indeed the crash is happening deep within FNET. It's going to take some serious work to unravel to figure out what the actual cause really is.
Just to be realistic, this is very unlikely to get fixed in 1.54. So many other things are updated & improved, and it's been just over 1 year since 1.53 and Arduino 1.8.15 is now 7 weeks old. We really need to wrap up 1.54 now.
But I definitely will be diving into this problem in the coming weeks. It will get fixed, though a problem this complex isn't the sort of thing that (typically) gets solved quickly.
Here are trimmed copies of the code to reproduce the problem. I sped it up quite a lot, so it usually crashes in less than 1 minute.
.......
I'm happy to say the new fault handler and CrashReport are working nicely. It does reboot and CrashReport give an address of the crash that's somewhere inside FNET/src/stack/fnet_tcp.c.
.I noticed that if I make the server to send bigger payloads the problem does not occur. Also if I use connection keep-alive header and do not close the socket there is no problems. So it seems that the problem is related to opening new connectiona repeatedly. Do you need some more information? For example if you want I can try to reproduce the problem with a programming language of your choice.
@vjmuzik, which version of FNET does it use? The newest?
I've been looking into this NativeEthernet bug. Indeed the crash is happening deep within FNET. It's going to take some serious work to unravel to figure out what the actual cause really is.
Just to be realistic, this is very unlikely to get fixed in 1.54. So many other things are updated & improved, and it's been just over 1 year since 1.53 and Arduino 1.8.15 is now 7 weeks old. We really need to wrap up 1.54 now.
But I definitely will be diving into this problem in the coming weeks. It will get fixed, though a problem this complex isn't the sort of thing that (typically) gets solved quickly.
......
platform_packages =
framework-arduinoteensy@https://github.com/maxgerhardt/teensy-core-pio-package.git#1.54-beta12
github.com/PaulStoffregen/LittleFS/pull/19 for 6 added LittleFS Integrity sketches.
Just tested each on Media - and on fail finally added while(1);