NativeEthernet with x.509 cert

CeKl

Member
Hi everyone,

I have been trying for some time now to realize the following via the NativeEthernet-Libary and a Teensy 4.1 with the Ethernet-Kit:

The Teensy 4.1 regularly pings a web server via https GET request and receives a control command back. However, our computing center has given us the requirement to store an x.509 certificate on the Teensy so that the proxy server in the network forwards the request outside. Alternatively, if the whole thing is without a certificate, then an SSH connection is ok with them, but this would mean rewriting or creating a translate layer for the backend.

Now to my question:
Has anyone already gotten the NativeEthernet-Libary and a x.509 certificate to work? I last tried it with ArduinoBearSSL in the small demo below, but the Teensy fails when calling the connection (sslClient.connect()) and reboots.

If I can help with more information, feel free to ask!
Many thanks in advance!


Hardware:
  • Teensy 4.1
  • Ethernet-Kit (soldered and connected according to explanation; works without SSL)
Environment:
  • Arduino 2.3.3
  • Windows 11

C++:
#include <NativeEthernet.h>
#include <ArduinoBearSSL.h>


// Define the server and port
const char *server = "api.domain.com";  // Replace with your server's address
const int server_port = 443;              // HTTPS port


IPAddress ip(192, 168, 0, 100);
IPAddress myDns(192, 168, 0, 1);


const uint8_t root_ca_der[] = {
  0x30, 0x82, 0x06, 0x03, 0x30, 0x82, 0x03, 0xeb, 0xa0, 0x03, 0x02, 0x01,
    ....
  0xb9, 0x5f, 0x49, 0xba, 0x56, 0xb0, 0x9b
};

const size_t root_ca_der_len = 1543;

EthernetClient ethClient;
BearSSLClient sslClient(ethClient);

byte mac[] = {};

unsigned long beginMicros, endMicros;
unsigned long byteCount = 0;
bool printWebData = true;


void teensyMAC(uint8_t *mac) {
  for (uint8_t by = 0; by < 2; by++) mac[by] = (HW_OCOTP_MAC1 >> ((1 - by) * 8)) & 0xFF;
  for (uint8_t by = 0; by < 4; by++) mac[by + 2] = (HW_OCOTP_MAC0 >> ((3 - by) * 8)) & 0xFF;
  Serial.printf("MAC: %02x:%02x:%02x:%02x:%02x:%02x\n", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
}


void setup() {
  Serial.begin(9600);
  while (!Serial) {
    ;
  }

  teensyMAC(mac);

  Serial.println("Initialize Ethernet with DHCP:");
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");

    // Check for Ethernet hardware present
    if (Ethernet.hardwareStatus() == EthernetNoHardware) {
      Serial.println("Ethernet shield was not found.  Sorry, can't run without hardware. :(");
      while (true) {
        delay(1);  // do nothing, no point running without Ethernet hardware
      }
    }
    if (Ethernet.linkStatus() == LinkOFF) {
      Serial.println("Ethernet cable is not connected.");
    }

    // try to congifure using IP address instead of DHCP:
    Ethernet.begin(mac, ip, myDns);
  } else {
    Serial.print("  DHCP assigned IP ");
    Serial.println(Ethernet.localIP());
  }


  // give a second to initialize:
  delay(1000);
  Serial.print("connecting to ");
  Serial.print(server);
  Serial.println("...");

  // CreateBearSSL certificate object
  br_x509_certificate root_ca_cert;
  memset(&root_ca_cert, 0, sizeof(root_ca_cert));
  root_ca_cert.data = (unsigned char *)root_ca_der;
  root_ca_cert.data_len = root_ca_der_len;

  sslClient.setEccCert(root_ca_cert);

  // Send an HTTPS GET request
  if (sslClient.connect(server, server_port)) {
    sslClient.println("GET /status HTTP/1.1");
    sslClient.print("Host: ");
    sslClient.println(server);
    sslClient.println("Connection: close");
    sslClient.println();
  } else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
  beginMicros = micros();

}

void loop() {

  // if there are incoming bytes available
  // from the server, read them and print them:
  int len = sslClient.available();
  if (len > 0) {
    byte buffer[80];
    if (len > 80) len = 80;
    sslClient.read(buffer, len);
    if (printWebData) {
      Serial.write(buffer, len);  // show in the serial monitor (slows some boards)
    }
    byteCount = byteCount + len;
  }

  // if the server's disconnected, stop the client:
  if (!sslClient.connected()) {
    endMicros = micros();
    Serial.println();
    Serial.println("disconnecting.");
    sslClient.stop();
    Serial.print("Received ");
    Serial.print(byteCount);
    Serial.print(" bytes in ");
    float seconds = (float)(endMicros - beginMicros) / 1000000.0;
    Serial.print(seconds, 4);
    float rate = (float)byteCount / seconds / 1000.0;
    Serial.print(", rate = ");
    Serial.print(rate);
    Serial.print(" kbytes/second");
    Serial.println();

    // do nothing forevermore:
    while (true) {
      delay(1);
    }
  }
}
 
I am not an Ethernet expert, but you should try using QNEthernet instead.
It would appear that NativeEthernet is no longer maintained, whilst QNEthernet is actively maintained.
 
QNEthernet has support for Mbed TLS 2.x. There’s an MbedTLSDemo example. However, the use is a little complex, so I’m thinking about how to incorporate v3.x and how to simplify its use. No promises on any sort of timeline.
 
I implemented a layer over MbedTLS, but I haven’t completed things like unit tests or additions to the Readme or how-to instructions for install and use.

If you’re interested in the code, it’s here in this branch:

Basically, install MbedTLS 3.6.x, use the config from the MbedTLS example in your MbedTLS install, then wrap an EthernetClient in an MbedTLSClient. There are options for using certificates, but I don’t have time at the moment to write out anything more detailed.
 
Hi everyone,

Thanks for the responses. I tried a few approaches using mbedTLS and QNEthernet in the last weeks but failed, even with the MbedTLSDemo example. The client disconnects without a data count, and the host is resolved correctly :/

I looked at the example using the original QNEthernet with mbedTLS 2.28.9. Therefore, the function “mbedtls_hardware_poll” in mbedtls_entropy.c is defined, but I failed to integrate it. Maybe someone has an idea here? I set everything up as described in the header and the README, which leads to the need for mbedtls_hardware_poll, which is called, but I have not found a definition.

With the approach of @shawn and mbedTLS 3.6.X, I failed on the line “#include <qnethernet/MbedTLSClient.h>” since I installed QNEthernet in its newest version and cannot find “ MbedTLSClient.h” inside of it.


I think there is an error somewhere that disturbs everything, but I can't find it. Maybe someone here has an idea or a tip? Or maybe someone has a working example to share?

C++:
// SPDX-FileCopyrightText: (c) 2023-2024 Shawn Silverman <shawn@pobox.com>
// SPDX-License-Identifier: AGPL-3.0-or-later

// MbedTLSDemo shows how to use Mbed TLS 2.x.x.
//
// Prerequisites:
// 1. Enable the following lwIP options (in lwipopts.h or the project
//    build options):
//    1. LWIP_ALTCP
//    2. LWIP_ALTCP_TLS
//    3. LWIP_ALTCP_TLS_MBEDTLS
// 2. Enable the QNETHERNET_ALTCP_TLS_ADAPTER option.
//    (In the qnethernet_opts.h library file or the project build options.)
// 3. Install the latest 2.x.x version of Mbed TLS. As of this
//    writing, the current version is 2.28.9. In order to use this
//    example program in the Arduino IDE, Mbed TLS needs to be
//    installed as a library. Please see the README for more
//    information.
// 4. Update src/mbedtls/config.h in the Mbed TLS library with the desired
//    options. A sample config is in this project. Simply cut & paste.
//
// This file is part of the QNEthernet library.

#include <QNEthernet.h>
#if LWIP_ALTCP
#if LWIP_ALTCP_TLS && LWIP_ALTCP_TLS_MBEDTLS
#include <mbedtls.h>  // Must be included before referencing other mbedtls headers
#endif  // LWIP_ALTCP_TLS && LWIP_ALTCP_TLS_MBEDTLS
#endif  // LWIP_ALTCP

using namespace qindesign::network;

constexpr uint32_t kDHCPTimeout = 15'000;  // 15 seconds

// Connection information
#define HOST "www.google.com"
constexpr char kHost[]{HOST};
constexpr char kRequest[]{
    "HEAD / HTTP/1.1\r\n"
    "Host: " HOST "\r\n"
    "Connection: close\r\n"
    "\r\n"
};
constexpr uint16_t kPort = 443;   // HTTPS generally uses port 443

EthernetClient client;

bool disconnectedPrintLatch = false;  // Print "disconnected" only once
size_t dataCount = 0;

// Program setup.
void setup() {
  Serial.begin(115200);
  while (!Serial && millis() < 4000) {
    // Wait for Serial
  }
  printf("Starting...\r\n");

  uint8_t mac[6];
  Ethernet.macAddress(mac);  // This is informative; it retrieves, not sets
  printf("MAC = %02x:%02x:%02x:%02x:%02x:%02x\r\n",
         mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);

  // Get an IP address
  printf("Starting Ethernet with DHCP...\r\n");
  if (!Ethernet.begin()) {
    printf("Failed to start Ethernet\r\n");
    return;
  }
  if (!Ethernet.waitForLocalIP(kDHCPTimeout)) {
    printf("Failed to get IP address from DHCP\r\n");
    return;
  }

  IPAddress remoteIP;
  if (Ethernet.hostByName(kHost, remoteIP)) {
    printf("DNS resolved: %s -> %u.%u.%u.%u\r\n", kHost, remoteIP[0], remoteIP[1], remoteIP[2], remoteIP[3]);
  } else {
    printf("DNS resolution failed for %s\r\n", kHost);
  }



  IPAddress ip = Ethernet.localIP();
  printf("    Local IP    = %u.%u.%u.%u\r\n", ip[0], ip[1], ip[2], ip[3]);
  ip = Ethernet.subnetMask();
  printf("    Subnet mask = %u.%u.%u.%u\r\n", ip[0], ip[1], ip[2], ip[3]);
  ip = Ethernet.gatewayIP();
  printf("    Gateway     = %u.%u.%u.%u\r\n", ip[0], ip[1], ip[2], ip[3]);
  ip = Ethernet.dnsServerIP();
  printf("    DNS         = %u.%u.%u.%u\r\n", ip[0], ip[1], ip[2], ip[3]);

  // Connect and send the request
  printf("Connecting and sending request...\r\n");
  if (!client.connect(kHost, kPort)) {
    printf("Failed to connect\r\n");
    disconnectedPrintLatch = true;
  } else {
    client.writeFully(kRequest);
    client.flush();
    dataCount = 0;
    printf("[Awaiting response...]\r\n");
  }
}

// Main program loop.
void loop() {
  // Read the response
  if (client.connected()) {
    int avail = client.available();
    if (avail > 0) {
      dataCount += avail;
      for (int i = 0; i < avail; i++) {
        putc(client.read(), stdout);
      }
    }
  } else {
    if (!disconnectedPrintLatch) {
      disconnectedPrintLatch = true;
      printf("[Client disconnected]\r\n"
             "[Data count = %zu]\r\n", dataCount);
    }
  }
}
 

Attachments

  • config.h
    9.9 KB · Views: 13
  • config.h
    9.9 KB · Views: 15
  • lwipopts.h
    23.5 KB · Views: 14
  • qnethernet_opts.h
    3.6 KB · Views: 14
To use MbedTLSClient, you need to use the tlsclient branch of the QNEthernet library. Is this the version you’re using?
 
To use MbedTLSClient, you need to use the tlsclient branch of the QNEthernet library. Is this the version you’re using?
Thanks for your quick response. I tried to remove and reinstall QNEthernet from the tlsclient branch. Therefore, I get the compiling error of MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET, MBEDTLS_ERR_SSL_RECEIVED_EARLY_DATA, mbedtls_ssl_is_handshake_over, mbedtls_ssl_conf_psk_cb, mbedtls_ssl_set_hs_psk, and mbedtls_ssl_conf_psk were not declared in this scope, unfortunately inside of MbedTLSServer.cpp and MbedTLSClient.cpp.

I use QNEtherent now in the last version of tlsclient branch and mbedTLS in 3.6.3
 
Sure, I copied the content of the MbedTLS example at src/mbedtls/config.h
 

Attachments

  • config.h
    21 KB · Views: 14
Hmm. Let me come up with a set of instructions as if I was doing this from scratch. Perhaps there’s a step I don’t realize is missing. It won’t be right away, however. Next time I’m near my computational instruments.

I might be incorrect, but “config.h” doesn’t sound like the right file.
 
Hmm. Let me come up with a set of instructions as if I was doing this from scratch. Perhaps there’s a step I don’t realize is missing. It won’t be right away, however. Next time I’m near my computational instruments.

I might be incorrect, but “config.h” doesn’t sound like the right file.
Thank you very much. Most of it is new to me regarding the TLS setup with QNEthernet, and I am more than grateful for any ideas you have as soon as you have time!
 
@shawn - looks like I tested this to work on 12/17/24 - that was on pumble thread - the steps there now locked after 90 days on free version :(

Below is the comment in the sketch used then : MbedTLSDemo.ino

It has this sample_mbedtls_config.h in the sketch folder. Not sure of the steps you had me follow - but maybe this helps?

Code:
// MbedTLSDemo shows how to use Mbed TLS 3.6.x.
//
// Prerequisites:
// 1. Install the latest 3.6.x version of Mbed TLS. As of this
//    writing, the current version is 3.6.2. In order to use this
//    example program in the Arduino IDE, Mbed TLS needs to be
//    installed as a library. Please see the README for more
//    information.
// 2. Update src/mbedtls/mbedtls_config.h in the Mbed TLS library with
//    the desired options. A sample config is in this project. Simply
//    cut & paste.
//
// This file is part of the QNEthernet library.
 

Attachments

  • sample_mbedtls_config.h
    19.9 KB · Views: 15
I updated the Mbed TLS install section of the Readme (tlsclient QNEthernet branch). You need branch v3.6.3 of mbedtls. There's no config.h; it's mbedtls_config.h.
 
Back
Top