modulatedthreat
Member
I am trying to implement this websocket library found at https://github.com/Links2004/arduinoWebSockets on a Teensy 3.2 with a wiz850.
I am using Pauls latest Ethernet Library for the W5500 with the spi at 30 Mhz with 4K buffers uncommented.
It appears that most of the development on this library was done using an esp8266 or enc28j60, although the W5100 is also supposed to work.
To make things a little cleaner, I removed all the #if defined stuff that does not apply to my case using unifdef.
I am compiling and uploading using platformio, and getting it to compile with the linker flag -specs=nosys.specs.
I am having issues with the way the library uses server.available(). I will get to that but first, you need to know how the WebSockets class and structures are laid out.
WebSockets.h
and The WebSocketServer Class header..
And the WebSocketClient Header..
The sketch Im running on the Teensy that implements just the WebSocketServer and simple callback.
So now youre probly wondering what myWebSocket.loop() looks like. It calls 2 functions
void WebSocketsServer::loop(void) {
handleNewClients();
handleClientData();
}
the first problem I am having is when handleNewClients() is called. The functions dealing with that look like this:
So back to server.available().
The first issue is that "new WEBSOCKETS_NETWORK_CLASS(_server->available());" is returning a valid pointer even when _server->available() returns false and there is NO data coming into that port. So in the debug output it it loops over and over with:
[WS-Server][0] new client
[WS-Server][0] client connection lost.
[WS-Server][0] client disconnected.
I can stop this behavior by putting everything in handleNewClients() after the first line, "bool ok = false;" in an if statement: if(_server->available()) { }. But that cripples other functionality, like stopping all communication from the Teensy to the wiznet if the remote client disconnects, so I would like to use it as close to as is as possible.
To my understanding:
handleNewClients() should take an incomming client, IF ONE EXISTS, and send a pointer pointing to it through newClient().
The WSclient_t structure contains a member WEBSOCKETS_NETWORK_CLASS * tcp, that ends up being a pointer to an object of class EthernetClient from Pauls Ethernet library. In newClient() the pointer found in handleNewClients() is set equal to * tcp.
So my first question is Why does new WEBSOCKETS_NETWORK_CLASS(_server->available()) Return a valid pointer even when _server->available() returns false ?
My Second question is with such a complete library, would it make sense to merge this library with the ethernet library, similiar to how UDP is built in? With so many people using teensy for IOT stuff, and WebSocket capability being standard in modern browsers, Built in WebSockets would be SUPER attractive compared to HTTP Get requests and AJAX for browser based UI's written in javascript and html .
I am using Pauls latest Ethernet Library for the W5500 with the spi at 30 Mhz with 4K buffers uncommented.
It appears that most of the development on this library was done using an esp8266 or enc28j60, although the W5100 is also supposed to work.
To make things a little cleaner, I removed all the #if defined stuff that does not apply to my case using unifdef.
I am compiling and uploading using platformio, and getting it to compile with the linker flag -specs=nosys.specs.
I am having issues with the way the library uses server.available(). I will get to that but first, you need to know how the WebSockets class and structures are laid out.
WebSockets.h
Code:
#ifndef WEBSOCKETS_H_
#define WEBSOCKETS_H_
#include <Arduino.h>
#include <functional>
#define DEBUG_WEBSOCKETS(...) Serial.printf( __VA_ARGS__ )
#ifndef DEBUG_WEBSOCKETS
#define DEBUG_WEBSOCKETS(...)
#define NODEBUG_WEBSOCKETS
#endif
#define WEBSOCKETS_MAX_DATA_SIZE (15*1024)
// moves all Header strings to Flash
#define WEBSOCKETS_SAVE_RAM
#define WEBSOCKETS_TCP_TIMEOUT (2000)
#define NETWORK_ESP8266_ASYNC (0)
#define NETWORK_ESP8266 (1)
#define NETWORK_W5100 (2)
#define NETWORK_ENC28J60 (3)
// max size of the WS Message Header
#define WEBSOCKETS_MAX_HEADER_SIZE (14)
#include <Ethernet.h>
#include <SPI.h>
#define WEBSOCKETS_NETWORK_CLASS EthernetClient
#define WEBSOCKETS_NETWORK_SERVER_CLASS EthernetServer
// moves all Header strings to Flash (~300 Byte)
#ifdef WEBSOCKETS_SAVE_RAM
#define WEBSOCKETS_STRING(var) F(var)
#else
#define WEBSOCKETS_STRING(var) var
#endif
typedef enum {
WSC_NOT_CONNECTED,
WSC_HEADER,
WSC_CONNECTED
} WSclientsStatus_t;
typedef enum {
WStype_ERROR,
WStype_DISCONNECTED,
WStype_CONNECTED,
WStype_TEXT,
WStype_BIN,
WStype_FRAGMENT_TEXT_START,
WStype_FRAGMENT_BIN_START,
WStype_FRAGMENT,
WStype_FRAGMENT_FIN,
} WStype_t;
typedef enum {
WSop_continuation = 0x00, ///< %x0 denotes a continuation frame
WSop_text = 0x01, ///< %x1 denotes a text frame
WSop_binary = 0x02, ///< %x2 denotes a binary frame
///< %x3-7 are reserved for further non-control frames
WSop_close = 0x08, ///< %x8 denotes a connection close
WSop_ping = 0x09, ///< %x9 denotes a ping
WSop_pong = 0x0A ///< %xA denotes a pong
///< %xB-F are reserved for further control frames
} WSopcode_t;
typedef struct {
bool fin;
bool rsv1;
bool rsv2;
bool rsv3;
WSopcode_t opCode;
bool mask;
size_t payloadLen;
uint8_t * maskKey;
} WSMessageHeader_t;
typedef struct {
uint8_t num; ///< connection number
WSclientsStatus_t status;
WEBSOCKETS_NETWORK_CLASS * tcp;
bool isSocketIO; ///< client for socket.io server
String cUrl; ///< http url
uint16_t cCode; ///< http code
bool cIsUpgrade; ///< Connection == Upgrade
bool cIsWebsocket; ///< Upgrade == websocket
String cSessionId; ///< client Set-Cookie (session id)
String cKey; ///< client Sec-WebSocket-Key
String cAccept; ///< client Sec-WebSocket-Accept
String cProtocol; ///< client Sec-WebSocket-Protocol
String cExtensions; ///< client Sec-WebSocket-Extensions
uint16_t cVersion; ///< client Sec-WebSocket-Version
uint8_t cWsRXsize; ///< State of the RX
uint8_t cWsHeader[WEBSOCKETS_MAX_HEADER_SIZE]; ///< RX WS Message buffer
WSMessageHeader_t cWsHeaderDecode;
String base64Authorization; ///< Base64 encoded Auth request
String plainAuthorization; ///< Base64 encoded Auth request
String extraHeaders;
bool cHttpHeadersValid; ///< non-websocket http header validity indicator
size_t cMandatoryHeadersCount; ///< non-websocket mandatory http headers present count
} WSclient_t;
class WebSockets {
protected:
typedef std::function<void(WSclient_t * client, bool ok)> WSreadWaitCb;
virtual void clientDisconnect(WSclient_t * client);
virtual bool clientIsConnected(WSclient_t * client);
virtual void messageReceived(WSclient_t * client, WSopcode_t opcode, uint8_t * payload, size_t length, bool fin);
void clientDisconnect(WSclient_t * client, uint16_t code, char * reason = NULL, size_t reasonLen = 0);
bool sendFrame(WSclient_t * client, WSopcode_t opcode, uint8_t * payload = NULL, size_t length = 0, bool mask = false, bool fin = true, bool headerToPayload = false);
void headerDone(WSclient_t * client);
void handleWebsocket(WSclient_t * client);
bool handleWebsocketWaitFor(WSclient_t * client, size_t size);
void handleWebsocketCb(WSclient_t * client);
void handleWebsocketPayloadCb(WSclient_t * client, bool ok, uint8_t * payload);
String acceptKey(String & clientKey);
String base64_encode(uint8_t * data, size_t length);
bool readCb(WSclient_t * client, uint8_t *out, size_t n, WSreadWaitCb cb);
virtual size_t write(WSclient_t * client, uint8_t *out, size_t n);
size_t write(WSclient_t * client, const char *out);
};
#endif /* WEBSOCKETS_H_ */and The WebSocketServer Class header..
Code:
#define WEBSOCKETS_SERVER_CLIENT_MAX (5)
class WebSocketsServer: protected WebSockets {
public:
typedef std::function<void (uint8_t num, WStype_t type, uint8_t * payload, size_t length)> WebSocketServerEvent;
typedef std::function<bool (String headerName, String headerValue)> WebSocketServerHttpHeaderValFunc;
WebSocketsServer(uint16_t port, String origin = "", String protocol = "arduino");
virtual ~WebSocketsServer(void);
void begin(void);
void loop(void);
void onEvent(WebSocketServerEvent cbEvent);
void onValidateHttpHeader(
WebSocketServerHttpHeaderValFunc validationFunc,
const char* mandatoryHttpHeaders[],
size_t mandatoryHttpHeaderCount);
bool sendTXT(uint8_t num, uint8_t * payload, size_t length = 0, bool headerToPayload = false);
bool sendTXT(uint8_t num, const uint8_t * payload, size_t length = 0);
bool sendTXT(uint8_t num, char * payload, size_t length = 0, bool headerToPayload = false);
bool sendTXT(uint8_t num, const char * payload, size_t length = 0);
bool sendTXT(uint8_t num, String & payload);
bool broadcastTXT(uint8_t * payload, size_t length = 0, bool headerToPayload = false);
bool broadcastTXT(const uint8_t * payload, size_t length = 0);
bool broadcastTXT(char * payload, size_t length = 0, bool headerToPayload = false);
bool broadcastTXT(const char * payload, size_t length = 0);
bool broadcastTXT(String & payload);
bool sendBIN(uint8_t num, uint8_t * payload, size_t length, bool headerToPayload = false);
bool sendBIN(uint8_t num, const uint8_t * payload, size_t length);
bool broadcastBIN(uint8_t * payload, size_t length, bool headerToPayload = false);
bool broadcastBIN(const uint8_t * payload, size_t length);
bool sendPing(uint8_t num, uint8_t * payload = NULL, size_t length = 0);
bool sendPing(uint8_t num, String & payload);
bool broadcastPing(uint8_t * payload = NULL, size_t length = 0);
bool broadcastPing(String & payload);
void disconnect(void);
void disconnect(uint8_t num);
void setAuthorization(const char * user, const char * password);
void setAuthorization(const char * auth);
protected:
uint16_t _port;
String _origin;
String _protocol;
String _base64Authorization; ///< Base64 encoded Auth request
String * _mandatoryHttpHeaders;
size_t _mandatoryHttpHeaderCount;
WEBSOCKETS_NETWORK_SERVER_CLASS * _server;
WSclient_t _clients[WEBSOCKETS_SERVER_CLIENT_MAX];
WebSocketServerEvent _cbEvent;
WebSocketServerHttpHeaderValFunc _httpHeaderValidationFunc;
bool newClient(WEBSOCKETS_NETWORK_CLASS * TCPclient);
void messageReceived(WSclient_t * client, WSopcode_t opcode, uint8_t * payload, size_t length, bool fin);
void clientDisconnect(WSclient_t * client);
bool clientIsConnected(WSclient_t * client);
void handleNewClients(void);
void handleClientData(void);
void handleHeader(WSclient_t * client, String * headerLine);
/**
* called if a non Websocket connection is coming in.
* Note: can be override
* @param client WSclient_t * ptr to the client struct
*/
virtual void handleNonWebsocketConnection(WSclient_t * client) {
DEBUG_WEBSOCKETS("[WS-Server][%d][handleHeader] no Websocket connection close.\n", client->num);
client->tcp->write("HTTP/1.1 400 Bad Request\r\n"
"Server: arduino-WebSocket-Server\r\n"
"Content-Type: text/plain\r\n"
"Content-Length: 32\r\n"
"Connection: close\r\n"
"Sec-WebSocket-Version: 13\r\n"
"\r\n"
"This is a Websocket server only!");
clientDisconnect(client);
}
/**
* called if a non Authorization connection is coming in.
* Note: can be override
* @param client WSclient_t * ptr to the client struct
*/
virtual void handleAuthorizationFailed(WSclient_t *client) {
client->tcp->write("HTTP/1.1 401 Unauthorized\r\n"
"Server: arduino-WebSocket-Server\r\n"
"Content-Type: text/plain\r\n"
"Content-Length: 45\r\n"
"Connection: close\r\n"
"Sec-WebSocket-Version: 13\r\n"
"WWW-Authenticate: Basic realm=\"WebSocket Server\""
"\r\n"
"This Websocket server requires Authorization!");
clientDisconnect(client);
}
/**
* called for sending a Event to the app
* @param num uint8_t
* @param type WStype_t
* @param payload uint8_t *
* @param length size_t
*/
virtual void runCbEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t length) {
if(_cbEvent) {
_cbEvent(num, type, payload, length);
}
}
/*
* Called at client socket connect handshake negotiation time for each http header that is not
* a websocket specific http header (not Connection, Upgrade, Sec-WebSocket-*)
* If the custom httpHeaderValidationFunc returns false for any headerName / headerValue passed, the
* socket negotiation is considered invalid and the upgrade to websockets request is denied / rejected
* This mechanism can be used to enable custom authentication schemes e.g. test the value
* of a session cookie to determine if a user is logged on / authenticated
*/
virtual bool execHttpHeaderValidation(String headerName, String headerValue) {
if(_httpHeaderValidationFunc) {
//return the value of the custom http header validation function
return _httpHeaderValidationFunc(headerName, headerValue);
}
//no custom http header validation so just assume all is good
return true;
}
private:
/*
* returns an indicator whether the given named header exists in the configured _mandatoryHttpHeaders collection
* @param headerName String ///< the name of the header being checked
*/
bool hasMandatoryHeader(String headerName);
};
#endif /* WEBSOCKETSSERVER_H_ */And the WebSocketClient Header..
Code:
class WebSocketsClient: private WebSockets {
public:
typedef std::function<void (WStype_t type, uint8_t * payload, size_t length)> WebSocketClientEvent;
WebSocketsClient(void);
virtual ~WebSocketsClient(void);
void begin(const char *host, uint16_t port, const char * url = "/", const char * protocol = "arduino");
void begin(String host, uint16_t port, String url = "/", String protocol = "arduino");
void beginSocketIO(const char *host, uint16_t port, const char * url = "/socket.io/?EIO=3", const char * protocol = "arduino");
void beginSocketIO(String host, uint16_t port, String url = "/socket.io/?EIO=3", String protocol = "arduino");
void loop(void);
void onEvent(WebSocketClientEvent cbEvent);
bool sendTXT(uint8_t * payload, size_t length = 0, bool headerToPayload = false);
bool sendTXT(const uint8_t * payload, size_t length = 0);
bool sendTXT(char * payload, size_t length = 0, bool headerToPayload = false);
bool sendTXT(const char * payload, size_t length = 0);
bool sendTXT(String & payload);
bool sendBIN(uint8_t * payload, size_t length, bool headerToPayload = false);
bool sendBIN(const uint8_t * payload, size_t length);
bool sendPing(uint8_t * payload = NULL, size_t length = 0);
bool sendPing(String & payload);
void disconnect(void);
void setAuthorization(const char * user, const char * password);
void setAuthorization(const char * auth);
void setExtraHeaders(const char * extraHeaders = NULL);
void setReconnectInterval(unsigned long time);
protected:
String _host;
uint16_t _port;
WSclient_t _client;
WebSocketClientEvent _cbEvent;
unsigned long _lastConnectionFail;
unsigned long _reconnectInterval;
void messageReceived(WSclient_t * client, WSopcode_t opcode, uint8_t * payload, size_t length, bool fin);
void clientDisconnect(WSclient_t * client);
bool clientIsConnected(WSclient_t * client);
void handleClientData(void);
void sendHeader(WSclient_t * client);
void handleHeader(WSclient_t * client, String * headerLine);
void connectedCb();
void connectFailedCb();
/**
* called for sending a Event to the app
* @param type WStype_t
* @param payload uint8_t *
* @param length size_t
*/
virtual void runCbEvent(WStype_t type, uint8_t * payload, size_t length) {
if(_cbEvent) {
_cbEvent(type, payload, length);
}
}
};
#endif /* WEBSOCKETSCLIENT_H_ */The sketch Im running on the Teensy that implements just the WebSocketServer and simple callback.
Code:
#include <SPI.h>
#include <Ethernet.h>
#include <WebSocketsServer.h>
byte debug = 1;
#define netCSpin 10 //SPI
#define netMOSIpin 11
#define netMISOpin 12
//#define netCLKpin 27
#define netCLKpin 13
#define netRSTpin 9
#define WebSocketPort 8081
byte mac[] = { 0x04, 0xE9, 0xE5, 0x04, 0xE6, 0x09 };
IPAddress ip(192, 168, 1, 4);
WebSocketsServer myWebSocket = WebSocketsServer(WebSocketPort);
void setup() {
pinMode(netRSTpin, OUTPUT);
pinMode(netCSpin, OUTPUT);
if (debug)
Serial.begin(115200);
SPI.begin();
wizReset();
Ethernet.begin(mac, ip);
if (debug)
Serial.println(Ethernet.localIP());
myWebSocket.begin();
myWebSocket.onEvent(webSocketEvent);
}
void loop() {
myWebSocket.loop();
}
void wizReset() {
digitalWrite(netRSTpin, LOW); // begin reset the WIZ820io
digitalWrite(netCSpin, HIGH); // de-select WIZ820io
digitalWrite(netRSTpin, HIGH); // end reset pulse
}
void webSocketEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t length) {
switch (type) {
case WStype_ERROR:
Serial.printf("[%u] ERROR!\n", num);
break;
case WStype_DISCONNECTED:
Serial.printf("[%u] Disconnected!\n", num);
break;
case WStype_CONNECTED:
Serial.println("Connected");
myWebSocket.sendTXT(num, "Connected!\n");
break;
case WStype_TEXT:
Serial.printf("[%u] get Text: %s\n", num, payload);
myWebSocket.sendTXT(num, "GOT message!! YAAAEEE");
break;
case WStype_BIN:
Serial.printf("[%u] got binary! \n", num);
break;
}
}So now youre probly wondering what myWebSocket.loop() looks like. It calls 2 functions
void WebSocketsServer::loop(void) {
handleNewClients();
handleClientData();
}
the first problem I am having is when handleNewClients() is called. The functions dealing with that look like this:
Code:
void WebSocketsServer::handleNewClients(void) {
bool ok = false;
WEBSOCKETS_NETWORK_CLASS * tcpClient = new WEBSOCKETS_NETWORK_CLASS(_server->available());
if (!tcpClient) {
DEBUG_WEBSOCKETS("[WS-Client] creating Network class failed!");
return;
}
ok = newClient(tcpClient);
if (!ok) {
// no free space to handle client
DEBUG_WEBSOCKETS("[WS-Server] no free space new client\n");
tcpClient->stop();
}
}
bool WebSocketsServer::newClient(WEBSOCKETS_NETWORK_CLASS * TCPclient) {
WSclient_t * client;
// search free list entry for client
for (uint8_t i = 0; i < WEBSOCKETS_SERVER_CLIENT_MAX; i++) {
client = &_clients[i];
// state is not connected or tcp connection is lost
if (!clientIsConnected(client)) {
client->tcp = TCPclient;
// set Timeout for readBytesUntil and readStringUntil
client->tcp->setTimeout(WEBSOCKETS_TCP_TIMEOUT);
client->status = WSC_HEADER;
DEBUG_WEBSOCKETS("[WS-Server][%d] new client\n", client->num);
return true;
break;
}
}
return false;
}
bool WebSocketsServer::clientIsConnected(WSclient_t * client) {
if(!client->tcp) {
return false;
}
if(client->tcp->connected()) {
if(client->status != WSC_NOT_CONNECTED) {
return true;
}
} else {
// client lost
if(client->status != WSC_NOT_CONNECTED) {
DEBUG_WEBSOCKETS("[WS-Server][%d] client connection lost.\n", client->num);
// do cleanup
clientDisconnect(client);
}
}
if(client->tcp) {
// do cleanup
DEBUG_WEBSOCKETS("[WS-Server][%d] client list cleanup.\n", client->num);
clientDisconnect(client);
}
return false;
}
void WebSocketsServer::clientDisconnect(WSclient_t * client) {
if(client->tcp) {
if(client->tcp->connected()) {
client->tcp->flush();
client->tcp->stop();
}
delete client->tcp;
client->tcp = NULL;
}
client->cUrl = "";
client->cKey = "";
client->cProtocol = "";
client->cVersion = 0;
client->cIsUpgrade = false;
client->cIsWebsocket = false;
client->cWsRXsize = 0;
client->status = WSC_NOT_CONNECTED;
DEBUG_WEBSOCKETS("[WS-Server][%d] client disconnected.\n", client->num);
runCbEvent(client->num, WStype_DISCONNECTED, NULL, 0);
}So back to server.available().
The first issue is that "new WEBSOCKETS_NETWORK_CLASS(_server->available());" is returning a valid pointer even when _server->available() returns false and there is NO data coming into that port. So in the debug output it it loops over and over with:
[WS-Server][0] new client
[WS-Server][0] client connection lost.
[WS-Server][0] client disconnected.
I can stop this behavior by putting everything in handleNewClients() after the first line, "bool ok = false;" in an if statement: if(_server->available()) { }. But that cripples other functionality, like stopping all communication from the Teensy to the wiznet if the remote client disconnects, so I would like to use it as close to as is as possible.
To my understanding:
handleNewClients() should take an incomming client, IF ONE EXISTS, and send a pointer pointing to it through newClient().
The WSclient_t structure contains a member WEBSOCKETS_NETWORK_CLASS * tcp, that ends up being a pointer to an object of class EthernetClient from Pauls Ethernet library. In newClient() the pointer found in handleNewClients() is set equal to * tcp.
So my first question is Why does new WEBSOCKETS_NETWORK_CLASS(_server->available()) Return a valid pointer even when _server->available() returns false ?
My Second question is with such a complete library, would it make sense to merge this library with the ethernet library, similiar to how UDP is built in? With so many people using teensy for IOT stuff, and WebSocket capability being standard in modern browsers, Built in WebSockets would be SUPER attractive compared to HTTP Get requests and AJAX for browser based UI's written in javascript and html .