Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 7 of 7

Thread: T4.1 NativeEthernet library question (WebServer/process a POST)

  1. #1
    Junior Member
    Join Date
    Dec 2021
    Posts
    6

    T4.1 NativeEthernet library question (WebServer/process a POST)

    Hi all,

    Recently migrating a project from an ESP to Teensy 4.1.
    My old code used ESP8266WebServer.h and now I'm moving over to NativeEthernet.

    Here's what I've been able to successfully do so far:
    • Get the Teensy on my network
    • Enable mDNS and a basic webserver
    • Get a client to connect; The Teensy is serving up my HTML/CSS/jScript and it renders great.
    • Looking at the serial output, the POST is coming through!


    What isn't really clear is how to process the data. I need the syntax to be able to get to my form data. I've been using the googles to try to find some documentation on the library, and have come up empty handed. In the ESP library, it was as simple as taking server.arg(0) (First form element value), server.arg(1) (Second Form element value), etc.

    So effectively I've got a form with a bunch UX elements that I want the user to interact with and when they "post" have the Teensy do it's thing (in this case, control a bunch of LEDs). I've also gotten the Teensy able to drive the LED's with different patterns, etc, so I got that part working as well... I just need to marry these together.

    If anyone has an example of handling a POST in the webserver and managing the form elements, I can take it from here.

    Thanks in advance,
    J

  2. #2
    Senior Member
    Join Date
    Mar 2017
    Location
    Oakland, CA, USA
    Posts
    394
    You might be looking for the “application/x-www-form-urlencoded” content type. Or perhaps even “multipart/form-data”. See: https://developer.mozilla.org/en-US/...P/Methods/POST

  3. #3
    Junior Member
    Join Date
    Dec 2021
    Posts
    6
    Quote Originally Posted by shawn View Post
    You might be looking for the “application/x-www-form-urlencoded” content type. Or perhaps even “multipart/form-data”. See: https://developer.mozilla.org/en-US/...P/Methods/POST
    Shawn, Firstly, thank you. I thought maybe forcing the form to text/plain might solve my problem.
    However, no matter what encoding method I try, It's not entirely obvious on how the POST data is getting brought back... I'm SURE it's coming back; but using client.read is clearly not the right method... This is where I'm struggling. I know someone must have the solution

    So I've pretty much given up with POST at this point.
    What I've been able to get working is using GET, although I really hate this as it just bangs everything in the URL... but at least I can parse this out now.

    Here's the response I get with "GET":
    Code:
    new client
    GET /payload?fname=Hi&lname=There HTTP/1.1 <= YES, I can get my form data here!!
    Host: daboosh.local
    Connection: keep-alive
    Upgrade-Insecure-Requests: 1
    ...
    Here are the various POST tests - I feel the data is "just out of reach"
    (multipart/form-data)
    Code:
    new client
    POST /payload HTTP/1.1
    Host: daboosh.local
    Connection: keep-alive
    Content-Length: 238 <= WHERE IS THE CONTENT?
    Cache-Control: max-age=0
    Upgrade-Insecure-Requests: 1
    Origin: http://daboosh.local
    Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryVrAPwAqlTfYBEvxE  <= WHERE IS THE CONTENT?
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
    ...
    (x-www-form-urlencoded)
    Code:
    new client
    POST /payload HTTP/1.1
    Host: daboosh.local
    Connection: keep-alive
    Content-Length: 22 <= WHERE IS THE CONTENT?
    Cache-Control: max-age=0
    Upgrade-Insecure-Requests: 1
    Origin: http://daboosh.local
    Content-Type: application/x-www-form-urlencoded
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
    ...
    (Good old text/plain)
    Code:
    new client
    POST /payload HTTP/1.1
    Host: daboosh.local
    Connection: keep-alive
    Content-Length: 25 <= WHERE IS THE CONTENT?
    Cache-Control: max-age=0
    Upgrade-Insecure-Requests: 1
    Origin: http://daboosh.local
    Content-Type: text/plain
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
    Referer: http://daboosh.local/payload
    ...
    And finally here is the test script (It's pretty much 100% "example WebServerMDNS" with the HTML swapped out for the form...)
    Code:
    #include "NativeEthernet.h"
    
    uint8_t mac[6];
    void teensyMAC(uint8_t *mac) {
        for(uint8_t by=0; by<2; by++) mac[by]=(HW_OCOTP_MAC1 >> ((1-by)*8)) & 0xFF;
        for(uint8_t by=0; by<4; by++) mac[by+2]=(HW_OCOTP_MAC0 >> ((3-by)*8)) & 0xFF;
        Serial.printf("MAC: %02x:%02x:%02x:%02x:%02x:%02x\n", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
    }
    
    //EthernetServer server(443, true); //Uncomment for TLS
    EthernetServer server(80);
    
    void setup() {;
      // put your setup code here, to run once:
      delay(3000);
      teensyMAC(mac);
      Ethernet.begin(mac);
      MDNS.begin("DaBoosh", 2); //.local Domain name and number of services
      MDNS.setServiceName("Teensy41_Service_Name"); //Uncomment to change service name
      MDNS.addService("_https._tcp", 443); //Uncomment for TLS
      MDNS.addService("_http._tcp", 80);
      server.begin();
      Serial.print("IP  address: ");
      Serial.println(Ethernet.localIP());
      Serial.send_now();
    }
    
    void loop() {
      // listen for incoming clients
      EthernetClient client = server.available();
      if (client) {
        Serial.println("new client");
        // an http request ends with a blank line
        boolean currentLineIsBlank = true;
        while (client.connected()) {
          if (client.available()) {
            char c = client.read();
            Serial.write(c);
            // if you've gotten to the end of the line (received a newline
            // character) and the line is blank, the http request has ended,
            // so you can send a reply
            if (c == '\n' && currentLineIsBlank) {
              // send a standard http response header
              client.println("HTTP/1.1 200 OK");
              client.println("Content-Type: text/html");
              client.println("Connection: close");  // the connection will be closed after completion of the response
              // client.println("Refresh: 5");  // refresh the page automatically every 5 sec
              client.println();
    
              // Here is the basic form information
              client.println("<!DOCTYPE html>\n");
              client.println("<html>\n");
              client.println("<body>\n");
              client.println("<h1>Testing the Forms</h1>\n");
              client.println("<p>Try a POST and GET</p>\n");
              client.println("<form action=\"/payload\" method=\"get\" target=\"_blank\">\n");
              client.println("  <label for=\"fname\">First name:</label>\n");
              client.println("  <input type=\"text\" id=\"fname\" name=\"fname\"><br><br>\n");
              client.println("  <label for=\"lname\">Last name:</label>\n");
              client.println("  <input type=\"text\" id=\"lname\" name=\"lname\"><br><br>\n");
              client.println("  <input type=\"submit\" value=\"Submit using GET\">\n");
              client.println("  <input type=\"submit\" formmethod=\"post\" formenctype=\"multipart/form-data\" value=\"Submit POST as Multipart/form-data\">\n");
              client.println("  <input type=\"submit\" formmethod=\"post\" formenctype=\"application/x-www-form-urlencoded\" value=\"Submit POST as x-www-form-urlencoded\">\n");
              client.println("  <input type=\"submit\" formmethod=\"post\" formenctype=\"text/plain\" value=\"Submit POST as text/plain\">");
              client.println("  <input type=\"submit\" formmethod=\"post\" value=\"Submit using POST\">\n");
              client.println("</form>\n");
              client.println("</body>\n");
              client.println("</html>");
             
              delay(500);
              client.close(); //If "Connection: close" make sure to close after printing and before stop to avoid harsh reset
              break;
            }
            if (c == '\n') {
              // you're starting a new line
              currentLineIsBlank = true;
            } else if (c != '\r') {
              // you've gotten a character on the current line
              currentLineIsBlank = false;
            }
          }
        }
        // give the web browser time to receive the data
        delay(500);
        // close the connection:
        client.stop();
        Serial.println("client disconnected");
      }
    }

  4. #4
    Junior Member
    Join Date
    Dec 2021
    Posts
    6
    Oh my goodness... I figured it out.

    The sample code makes the assumption that the client response is complete with a new line and a blank line, and was breaking before all the data could be written out.

    Simply by removing the break I'm able to see the data now

    Code:
    new client
    POST /payload HTTP/1.1
    Host: daboosh.local
    Connection: keep-alive
    Content-Length: 23
    Cache-Control: max-age=0
    Upgrade-Insecure-Requests: 1
    Origin: http://daboosh.local
    Content-Type: text/plain
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
    Referer: http://daboosh.local/payload
    Accept-Encoding: gzip, deflate
    Accept-Language: en-US,en;q=0.9
    
    fname=Hello&lname=World <=TADA. There it is!!! Content length is indeed 23 bytes
    client disconnected
    Hopefully this can help others out, and now I can code all this up properly.

  5. #5
    Senior Member
    Join Date
    Mar 2017
    Location
    Oakland, CA, USA
    Posts
    394
    Note: `println` doesn't necessarily guarantee that all the bytes are written. Technically, you need to loop until all the bytes are written. Additionally, the `print` and `println` calls don't always accurately return the number of bytes transmitted, depending on which version you're calling from which library. The best way to send data like this is actually using `write` calls. QNEthernet provides a `writeFully()` function that does this for you. (See https://forum.pjrc.com/threads/68389...l=1#post290568 for code, or copy the code from QNEthernetClient.cpp from QNEthernet.)

    Some additional notes:
    1. If you do choose to use the `writeFully()` approach instead, make sure to add the "\r\n" bytes where you want line breaks.
    2. The above point only applies to lines that need it, for example, HTTP headers. HTML doesn't actually need these. Their utility comes when you want to look at the page code and have it formatted a little nicer, rather than it being on one line. If formatting isn't important to you, you don't need the CRLF bytes.
    3. I would add a `flush()` call after all your data is fully written. Using some delay isn't a guarantee that all the data has actually gone out, unless the delay time is sufficiently long, but that depends on a lot of factors. 500ms is probably enough, but it's uncertain.

    By "removing the break," did you mean that you removed the `break;` line after the `client.close()` call?
    Last edited by shawn; 01-01-2022 at 12:27 AM.

  6. #6
    Junior Member
    Join Date
    Dec 2021
    Posts
    6
    Quote Originally Posted by shawn View Post
    Note: `println` doesn't necessarily guarantee that all the bytes are written. Technically, you need to loop until all the bytes are written. Additionally, the `print` and `println` calls don't always accurately return the number of bytes transmitted, depending on which version you're calling from which library. The best way to send data like this is actually using `write` calls. QNEthernet provides a `writeFully()` function that does this for you. (See https://forum.pjrc.com/threads/68389...l=1#post290568 for code, or copy the code from QNEthernetClient.cpp from QNEthernet.)

    Some additional notes:
    1. If you do choose to use the `writeFully()` approach instead, make sure to add the "\r\n" bytes where you want line breaks.
    2. The above point only applies to lines that need it, for example, HTTP headers. HTML doesn't actually need these. Their utility comes when you want to look at the page code and have it formatted a little nicer, rather than it being on one line. If formatting isn't important to you, you don't need the CRLF bytes.
    3. I would add a `flush()` call after all your data is fully written. Using some delay isn't a guarantee that all the data has actually gone out, unless the delay time is sufficiently long, but that depends on a lot of factors. 500ms is probably enough, but it's uncertain.

    By "removing the break," did you mean that you removed the `break;` line after the `client.close()` call?
    Thank you very much for the tips on QNEthernet.
    Removing the break after client.close: EXACTLY correct; I just commented the line. It allowed the form data to come through... which happens AFTER a blank line/new line. (Note the blank line after "Accept-Language: en-US,en;q=0.9"). I'm surprised I didn't spot this earlier.

    So I am currently able to grab, parse, and assign the form data to variables perfectly. HOWEVER, I now have another issue. fastLED processing breaks when I invoke Ethernet.begin but I'll start a new thread for that. This has been a bit of a struggle migrating from ESP...

  7. #7
    Member
    Join Date
    Jan 2020
    Location
    Toronto, Canada
    Posts
    74
    You can have a look at my EthernetWebServer Library, which supports QNEthernet with these examples in Examples for QNEthernet

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •