T4.1 Ethernet Library

Update:
Code: 
[B]Add MDNS wrapper[/B]
-Add WebServerMDNS example
-Update keywords

For a web server all it takes is two lines to use MDNS and get a .local address:
Code: 
MDNS.begin("Teensy41");  //Domain name
MDNS.addService("_http._tcp", 80);  //Type of service and port number
 
Hi vjmuzik,

thank you very much for doing this nice work....
Is the library able to do multicast ?

Thank you

Torsten
 
Native Ethernet Update:
Code: 
[B]Support for FNET TLS sockets (server-side only for now)[/B]
-Update WebServerMDNS with optional TLS

FNET Update:
Code: 
[B]Readded mbedtls source and support[/B]
-Fixed MDNS service name overflow

Most likely will work on client-side TLS tomorrow, I've already went through the mbedtls source and put everything in FLASHMEM and large constants in PROGMEM. Without doing that dynamic memory was at 54%, now it sits at 26% with TLS support and at 22% without TLS support, so not too much of an impact at all.
 
Native Ethernet Update:
Code: 
[B]Add client-side TLS support[/B]
-Add WebClientRepeatingTLS example
-Add client.close() to support proper HTTP close before disconnecting
-Add functions to set client and server certificates/keys (if needed)

FNET Update:
Code: 
[B]Change mbedtls client verify to optional[/B]
-Disable unused IPv6 support

As stated yesterday here is the client-side support for TLS as well as an example showcasing the PJRC homepage in all it's HTTPS glory.
 
I have started porting one of our application that has a web interface. It worked no issues on the Teensy 3.2, and 4.0 with the Wiznet Module, but it is having issues with the 4.1 Network Stack.

I am using the NativeEthernet library and an updated version of the Webuino library. From what I can tell, there may be two issues, 1. When writing out to the EhternetClient in response to a Web Request, the write is not blocking like it did on the wiznet. On the receiving side, I am seeing parts of my web page, but can tell where it is breaking up, like it was sending data, then interrupted by another write.

The second issue seems to be with the Client Flush Command, this appears to always hang forever.

-Steve
 
Can I get a minimal example of this so that I can find out what's causing the issue? Indeed FNET is non-blocking but it shouldn't send things out of order or overlapping since that would break TCP sockets altogether, I haven't really messed with the Client Flush Command so it very well may be a simple fix for that part.
 
Just for testing, I just modified the code where it was actually sending out the data to the web client. I placed a delay(10) in front of each Client.write(buffer, sizeof buffer)) and I get a complete web page now.

The issue may be cause by the size of the web page, it is quite large. The WebServer Application buffers the output into chunks, and sends them out when the buffer gets full (around 64 bytes). I have not been able to track down where in the HTML it is breaking, but I see several chunks, then a chunk that was interrupted, then more chunks. The complete web page is around 40K with inline css, images, etc.
 
What's likely happening is because of the non blocking nature of FNET it's completely filling the buffers before a packet is sent so data is being overwritten, that would be why adding a delay works since it gives it time to send. I'll work on a fix for this later likely involving client.flush since that's what it's supposed to stop from happening.
 
Just did another test with the examples included with the NativeEthernet Library.

I just added the following lines to the WebServer Example and see where my data is getting dropped.

I only get lines 0-19 and line 19 is a partial line. I don't get the ending br or closing html tag.



Code: 
          for (int analogChannel = 0; analogChannel < 6; analogChannel++) {
            int sensorReading = analogRead(analogChannel);
            client.print("analog input ");
            client.print(analogChannel);
            client.print(" is ");
            client.print(sensorReading);
            client.println("<br />");
          }

          >>for (int i = 0; i < 50; i++) {
          >>     client.printf("%5.5d - AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA<br>",i);  
          >>}
          client.print("<br />");
          
          client.println("</html>");
          break;
        }
 
Alright I posted the fix, it was simple, I also tested client.flush() and it does work without freezing, but that's probably because I already fixed the socketSendAvailable function that it uses in a recent update. Of course the problem was a buffer overflow like I thought since increasing the socket size let it send more data before failing, I tested the fix down to a 512 byte socket size with no issues so I would call it good.
 
That did help and I am now getting a complete web page. I am having some issues with stability and having the entire CPU lock up. I'm hoping it is a memory leak on my side, still investigating that one. The other thing I did notice, if I power up the board with no network cable connected, the board hangs initializing the Ethernet library. I'm still trying to tack down exactly which call is hanging. I wish we had a real debugger in Arduino!!
 
It doesn't hang but it certainly is blocking, I'm pretty sure the original w5500 fails if there is no cable connected on startup so I mirrored that in my begin function since the wrapper library is supposed to act the same way as the original. I suppose since I'm already extending the original api to support more things I should add a non blocking begin function so that hot plugging can be better supported.
 
I know this thread is really just regarding the NativeEthernet library, but could I ask if someone point me in the right direction, since I purchased my T4.1 I have wanted to try using the Ethernet for connecting my T4.1 to my laptop (no internet just direct) so that eventually I can use a web page to read from and be able to send data back to my T4.1. I have been visiting the 'www.startingelectronics.org' web site which has a good set of examples for pretty much a good starting point for what I am looking to do, albeit its for Arduino, and all I have had to changed is from ethernet.h to NativeEthernet.h, changed the SD card to 'BuiltIn' and got it to find the T4,.1 Mac address. Now I can run the basic webserver no problem but when I ask to send the web page from my SD card as well it doesn't display anything.

This works fine just sending Analog readings to a fairly blank web page;


Code: 
/*
  Web Server
 
 A simple web server that shows the value of the analog input pins.
 using an Arduino Wiznet Ethernet shield. 
 
 Circuit:
 * Ethernet shield attached to pins 10, 11, 12, 13
 * Analog inputs attached to pins A0 through A5 (optional)
 
 created 18 Dec 2009
 by David A. Mellis
 modified 9 Apr 2012
 by Tom Igoe
 
 */


#include <NativeEthernet.h>

// Enter a IP address for your controller below.

uint8_t mac[6];
IPAddress ip(192,168,1,177);

// Initialize the Ethernet server library
// with the IP address and port you want to use 
// (port 80 is default for HTTP):
EthernetServer server(80);

// Find Mac address for Teensy
static void teensyMAC(uint8_t *mac) {
  uint32_t m1 = HW_OCOTP_MAC1;
  uint32_t m2 = HW_OCOTP_MAC0;
  mac[0] = m1 >> 8;
  mac[1] = m1 >> 0;
  mac[2] = m2 >> 24;
  mac[3] = m2 >> 16;
  mac[4] = m2 >> 8;
  mac[5] = m2 >> 0;
}


void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }
  Serial.println("About to start Ethernet, apparently the cable needs plugged in before it will proceed!");
  teensyMAC(mac);
  static char teensyMacString[23];
  sprintf(teensyMacString, "MAC: %02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
  Serial.println(teensyMacString);
  
  // start the Ethernet connection and the server:
  Ethernet.begin(mac, ip);
  server.begin();
  Serial.print("server is at ");
  Serial.println(Ethernet.localIP());
}


void loop() {
  // listen for incoming clients
  EthernetClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");
          // add a meta refresh tag, so the browser pulls again every 5 seconds:
          client.println("<meta http-equiv=\"refresh\" content=\"1\">");
          // output the value of each analog input pin
          for (int analogChannel = 0; analogChannel < 6; analogChannel++) {
            int sensorReading = analogRead(analogChannel);
            client.print("analog input ");
            client.print(analogChannel);
            client.print(" is ");
            client.print(sensorReading);
            client.println("<br />");       
          }
          client.println("</html>");
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        } 
        else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
    Serial.println("client disonnected");
  }
}

But if I try to send the Web page from SD Card I get nothing

Code: 
/*--------------------------------------------------------------
  Program:      eth_websrv_SD_Ajax

  Description:  Arduino web server that serves up a web page
                that displays the status of a switch connected
                to pin 3 of the Arduino.
                The web page is stored on the SD card.
                The web page contains JavaScript code that uses
                Ajax to request the state of the switch every
                second.
  
  Hardware:     Arduino Uno and official Arduino Ethernet
                shield. Should work with other Arduinos and
                compatible Ethernet shields.
                2Gb micro SD card formatted FAT16
                Push button switch interfaced to pin 3 of the
                Arduino
                
  Software:     Developed using Arduino 1.0.5 software
                Should be compatible with Arduino 1.0 +
                SD card contains web page called index.htm
  
  References:   - WebServer example by David A. Mellis and 
                  modified by Tom Igoe
                - SD card examples by David A. Mellis and
                  Tom Igoe
                - Ethernet library documentation:
                  http://arduino.cc/en/Reference/Ethernet
                - SD Card library documentation:
                  http://arduino.cc/en/Reference/SD

  Date:         25 March 2013
  Modified:     17 June 2013
                - removed the use of the String class
 
  Author:       W.A. Smith, http://startingelectronics.org
--------------------------------------------------------------*/


#include <NativeEthernet.h>
#include <SD.h>

// size of buffer used to capture HTTP requests
#define REQ_BUF_SZ   400

// MAC address from Ethernet shield sticker under board
uint8_t mac[6];
IPAddress ip(192, 168, 1, 177); // IP address, may need to change depending on network
EthernetServer server(80);  // create a server at port 80
File webFile;

static void teensyMAC(uint8_t *mac) {
  uint32_t m1 = HW_OCOTP_MAC1;
  uint32_t m2 = HW_OCOTP_MAC0;
  mac[0] = m1 >> 8;
  mac[1] = m1 >> 0;
  mac[2] = m2 >> 24;
  mac[3] = m2 >> 16;
  mac[4] = m2 >> 8;
  mac[5] = m2 >> 0;
}
// Select SD Card
const int chipSelect = BUILTIN_SDCARD;  // For SD Card Chip select

char HTTP_req[REQ_BUF_SZ] = {0}; // buffered HTTP request stored as null terminated string
char req_index = 0;              // index into HTTP_req buffer

void setup()
{
    Serial.begin(9600);       // for debugging
  
  Serial.println("About to start Ethernet, apparently the cable needs plugged in before it will proceed!");
  teensyMAC(mac);
  static char teensyMacString[23];
  sprintf(teensyMacString, "MAC: %02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
  Serial.println(teensyMacString);   

    
    // initialize SD card
    Serial.println("Initializing SD card...");
    if (!SD.begin(chipSelect)) {
        Serial.println("ERROR - SD card initialization failed!");
        return;    // init failed
    }
    Serial.println("SUCCESS - SD card initialized.");
    // check for index.htm file
    if (!SD.exists("index.htm")) {
        Serial.println("ERROR - Can't find index.htm file!");
        return;  // can't find index file
    }
    Serial.println("SUCCESS - Found index.htm file.");
    pinMode(3, INPUT);        // switch is attached to Arduino pin 3
    
    Ethernet.begin(mac, ip);  // initialize Ethernet device
    server.begin();           // start to listen for clients
}

void loop()
{
    EthernetClient client = server.available();  // try to get client

    if (client) {  // got client?
        boolean currentLineIsBlank = true;
        while (client.connected()) {
            if (client.available()) {   // client data available to read
                char c = client.read(); // read 1 byte (character) from client
                // buffer first part of HTTP request in HTTP_req array (string)
                // leave last element in array as 0 to null terminate string (REQ_BUF_SZ - 1)
                if (req_index < (REQ_BUF_SZ - 1)) {
                    HTTP_req[req_index] = c;          // save HTTP request character
                    req_index++;
                }
                // last line of client request is blank and ends with \n
                // respond to client only after last line received
                if (c == '\n' && currentLineIsBlank) {
                    // send a standard http response header
                    client.println("HTTP/1.1 200 OK");
                    client.println("Content-Type: text/html");
                     client.println("Connection: keep-alive");
                    client.println();
                    // Ajax request
                    if (StrContains(HTTP_req, "ajax_switch")) {
                        // read switch state and send appropriate paragraph text
                        GetSwitchState(client);
                    }
                    else {  // web page request
                        // send web page
                        webFile = SD.open("index.htm");        // open web page file
                        if (webFile) {
                            while(webFile.available()) {
                                client.write(webFile.read()); // send web page to client
                            
                            }
                            webFile.close();
                        }
                    }
                    // display received HTTP request on serial port
                    Serial.println(HTTP_req);
                    // reset buffer index and all buffer elements to 0
                    req_index = 0;
                    StrClear(HTTP_req, REQ_BUF_SZ);
                    break;
                }
                // every line of text received from the client ends with \r\n
                if (c == '\n') {
                    // last character on line of received text
                    // starting new line with next character read
                    currentLineIsBlank = true;
                } 
                else if (c != '\r') {
                    // a text character was received from client
                    currentLineIsBlank = false;
                }
            } // end if (client.available())
        } // end while (client.connected())
        delay(1);      // give the web browser time to receive the data
        client.stop(); // close the connection
    } // end if (client)
}

// send the state of the switch to the web browser
void GetSwitchState(EthernetClient cl)
{
    if (digitalRead(3)) {
        cl.println("Switch state: ON");
    }
    else {
        cl.println("Switch state: OFF");
    }
}

// sets every element of str to 0 (clears array)
void StrClear(char *str, char length)
{
    for (int i = 0; i < length; i++) {
        str[i] = 0;
    }
}

// searches for the string sfind in the string str
// returns 1 if string found
// returns 0 if string not found
char StrContains(char *str, char *sfind)
{
    char found = 0;
    char index = 0;
    char len;

    len = strlen(str);
    
    if (strlen(sfind) > len) {
        return 0;
    }
    while (index < len) {
        if (str[index] == sfind[found]) {
            found++;
            if (strlen(sfind) == found) {
                return 1;
            }
        }
        else {
            found = 0;
        }
        index++;
    }

    return 0;
}

I can open the index.htm file from explorer and its fine so that code is ok and the SD card (2Gb) is working great. What am I overlooking, I am guessing something really stupid but its woods and trees at the moment.
Thanks
 
Have you tried serial printing the file as a starting point to verify everything is correct? You may have to update the libraries as well, I fixed a sending problem earlier today, and there will be some other fixes later tomorrow, but only for UDP.
 
Ok then that's interesting, it gets as far as "EthernetClient client = server.available();" constantly whereas I have put other Serial prints at each section it should enter when conditions are right.
The "192.168.1.177" is polling (I am using Chrome for this) issuing a statement that it is waiting then clears does nothing for several seconds then repeats it is waiting again. Does that mean I have some wrong with my laptop set-up...
 
If the basic web server worked then there shouldn’t be a problem with the laptop setup, I do my tests with a laptop as well so I know it does work directly connected to it just fine.
 
Just to mention I did your latest update yesterday just in case I had something corrupted.

Its really odd because any example I run that doesn't involve the sd card works fine, so I am guess two way conversation is taking place for them to work.
 
Thanks for that, I had a -17 as well but just once. I have just tried using the Web Datalogger sketch from the startingelectronics.org web site which reads an analog channel and stores it on the SD card but, the bit I am interested in, it sends a *.htm file from SD to browser. This works, its a bit more complicated (well for me it is), so I am thinking I need to work and understand this, I will get there.

For reference this is the Web Datalogger sketch.

Code: 
// Basic Arduino Web Server version 0.1 modified to include logging
// http://startingelectronics.org/software/arduino/web-server/basic-01/
//
// More details and web files available from:
// http://startingelectronics.org/software/arduino/web-server/01-log-data/
//
// Date: 11 July 2015
//

#include <NativeEthernet.h>
#include <SD.h>

// maximum length of file name including path
#define FILE_NAME_LEN  20
// HTTP request type
#define HTTP_invalid   0
#define HTTP_GET       1
#define HTTP_POST      2

// file types
#define FT_HTML       0
#define FT_ICON       1
#define FT_CSS        2
#define FT_JAVASCRIPT 3
#define FT_JPG        4
#define FT_PNG        5
#define FT_GIF        6
#define FT_TEXT       7
#define FT_INVALID    8


long log_time_ms = 5000; // how often to log data in milliseconds
long prev_log_time = 0;   // previous time log occurred

// the media access control (ethernet hardware) address for the shield:
const byte mac[] PROGMEM = { 0x04, 0xE9, 0xE5, 0x0B, 0xA9, 0x92 };
//the IP address for the shield:
const byte ip[] = { 192, 168, 1, 177 };  // does not work if this is put into Flash
// the router's gateway address:
const byte gateway[] PROGMEM = { 192, 168, 0, 1 };
// the subnet:
const byte subnet[] PROGMEM = { 255, 255, 255, 0 };
EthernetServer server(80);
// Select SD Card
const int chipSelect = BUILTIN_SDCARD;  // For SD Card Chip select

void setup() {
 
  Serial.begin(115200);       // for debugging
  
  if (!SD.begin(chipSelect)) {
    return;  // SD card initialization failed
  }

  Ethernet.begin((uint8_t*)mac, ip, gateway, subnet);
  server.begin();  // start listening for clients
}

void loop() {
  // if an incoming client connects, there will be bytes available to read:
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (ServiceClient(&client)) {
        // received request from client and finished responding
        break;
      }
    }  // while (client.connected())
    delay(1);
    client.stop();
  }  // if (client)
  // log the data
  LogData();
}// void loop()

void LogData(void)
{
  unsigned long current_time;   // current millisecond time
  File logFile;                 // file to log data to

  current_time = millis();  // get the current time in milliseconds
  if ((current_time - prev_log_time) > log_time_ms) {
    prev_log_time = current_time;
    // the log time has elapsed, so log another set of data
    logFile = SD.open("log.txt", FILE_WRITE);
    if (logFile) {
      logFile.print(current_time);
      logFile.print(" = ");
      logFile.println(analogRead(5)); // log the analog pin value
      logFile.close();
    }
  }
}

bool ServiceClient(EthernetClient *client)
{
  static boolean currentLineIsBlank = true;
  char cl_char;
  File webFile;
  // file name from request including path + 1 of null terminator
  char file_name[FILE_NAME_LEN + 1] = {0};  // requested file name
  char http_req_type = 0;
  char req_file_type = FT_INVALID;
  const char *file_types[] = {"text/html", "image/x-icon", "text/css", "application/javascript", "image/jpeg", "image/png", "image/gif", "text/plain"};
  
  static char req_line_1[40] = {0};  // stores the first line of the HTTP request
  static unsigned char req_line_index = 0;
  static bool got_line_1 = false;

  if (client->available()) {   // client data available to read
    cl_char = client->read();
    
    if ((req_line_index < 39) && (got_line_1 == false)) {
      if ((cl_char != '\r') && (cl_char != '\n')) {
        req_line_1[req_line_index] = cl_char;
        req_line_index++;
      }
      else {
        got_line_1 = true;
        req_line_1[39] = 0;
      }
    }
    
    if ((cl_char == '\n') && currentLineIsBlank) {
      // get HTTP request type, file name and file extension type index
      http_req_type = GetRequestedHttpResource(req_line_1, file_name, &req_file_type);
      if (http_req_type == HTTP_GET) {         // HTTP GET request
        if (req_file_type < FT_INVALID) {      // valid file type
          webFile = SD.open(file_name);        // open requested file
          if (webFile) {
            // send a standard http response header
            client->println(F("HTTP/1.1 200 OK"));
            client->print(F("Content-Type: "));
            client->println(file_types[req_file_type]);
            client->println(F("Connection: close"));
            client->println();
            // send web page
            while(webFile.available()) {
              int num_bytes_read;
              char byte_buffer[64];
              // get bytes from requested file
              num_bytes_read = webFile.read(byte_buffer, 64);
              // send the file bytes to the client
              client->write(byte_buffer, num_bytes_read);
            }
            webFile.close();
          }
          else {
            // failed to open file
          }
        }
        else {
          // invalid file type
        }
      }
      else if (http_req_type == HTTP_POST) {
        // a POST HTTP request was received
      }
      else {
        // unsupported HTTP request received
      }
      req_line_1[0] = 0;
      req_line_index = 0;
      got_line_1 = false;
      // finished sending response and web page
      return 1;
    }
    if (cl_char == '\n') {
      currentLineIsBlank = true;
    }
    else if (cl_char != '\r') {
      currentLineIsBlank = false;
    }
  }  // if (client.available())
  return 0;
}

// extract file name from first line of HTTP request
char GetRequestedHttpResource(char *req_line, char *file_name, char *file_type)
{
  char request_type = HTTP_invalid;  // 1 = GET, 2 = POST. 0 = invalid
  char *str_token;
  
  *file_type = FT_INVALID;
  
  str_token =  strtok(req_line, " ");    // get the request type
  if (strcmp(str_token, "GET") == 0) {
    request_type = HTTP_GET;
    str_token =  strtok(NULL, " ");      // get the file name
    if (strcmp(str_token, "/") == 0) {
      strcpy(file_name, "index.htm");
      *file_type = FT_HTML;
    }
    else if (strlen(str_token) <= FILE_NAME_LEN) {
      // file name is within allowed length
      strcpy(file_name, str_token);
      // get the file extension
      str_token = strtok(str_token, ".");
      str_token = strtok(NULL, ".");
      
      if      (strcmp(str_token, "htm") == 0) {*file_type = 0;}
      else if (strcmp(str_token, "ico") == 0) {*file_type = 1;}
      else if (strcmp(str_token, "css") == 0) {*file_type = 2;}
      else if (strcmp(str_token, "js")  == 0) {*file_type = 3;}
      else if (strcmp(str_token, "jpg") == 0) {*file_type = 4;}
      else if (strcmp(str_token, "png") == 0) {*file_type = 5;}
      else if (strcmp(str_token, "gif") == 0) {*file_type = 6;}
      else if (strcmp(str_token, "txt") == 0) {*file_type = 7;}
      else {*file_type = 8;}
    }
    else {
      // file name too long
    }
  }
  else if (strcmp(str_token, "POST") == 0) {
    request_type = HTTP_POST;
  }

  return request_type;
}

Can I say a big thank you for adapting this library for the T4.1, thanks.
 
@vjmuzik - I added the Ethernet kit to my T4.1's and tested with this library. All works well. Can you recommend a good FTP library to use. On my T3.6 I was using the Adafruit ESP8266 huzzah board with wifispi and wifiespi8266. Very slow transfer times. I was using both the arduino FTP Server library and FTP Client library trying to get a stable transfer. Not stable at all. Was using gFTP and FileZilla to transfer files. I have seen and used the tftp library with success but would like something more to what I was using before.
 
You must log in or register to reply here.
Back
Top