Packet Switched Networking

Status
Not open for further replies.

Camel

Well-known member
I've been toying with the idea of using a switched packet network for a robotics project. A switched packet network, as I understand it, is a network where all devices on the network connect to a switch device, and switches themselves may be connected to other switches. Each switch will have ports (just like an ethernet switch). Every device on the network has an address and this address is communicated to the switch so that it knows which address is available on which particular port. The device sends packets to the switch. Each packet contains source and destination addresses and a port number. The switch receives the packet and re-transmits it to the destination. In the device software, there are 'listeners' that listen on a particular port, and they get triggered whenever data is received on that port number.

I was thinking it would be implemented using UARTs (optionally with RS422/485 transceivers for robustness and range). Probably a 1Mb/s baud rate, maybe more. The switches would likely have to use DMA for maximum throughput.

The reason I'm kinda of interesting in doing this is because I don't know of any equivalent alternatives. There's ethernet, but it can be complex and requires special hardware and bulky switches and cables. There's SPI, I2C and whatnot but they are master/slave and don't have the speed or range. There's also CAN, which is awesome, but it's not well suited for high bandwidth data transfers and every packet is received by every node. In a switched network, bandwidth is only used when it needs to be so for the same bit rate you can get better throughput. Say you have devices A, B, C, and D plugged into a switch, if A blasts B with data, it will have little effect on the ability of C and D to communicate. Whereas if that happened on a CAN bus, C and D either wouldn't be able to talk or C would constantly interrupt A & B. I suppose ethernet is the closest equivalent that I know of, so I'm probably imagining something like ethernet 'lite' over UART with teensy switches.

At a higher level, I'm thinking there'd be two kinda modes of operation. There'd be datagrams and streams. Datagrams are limited to the max frame size and are sent without any need for acknowledgement. Like UDP. Streams emulate something like a UART. They are a two way connection between two devices on a particular port. Stream data is guaranteed to be received in order kind of like TCP. I'm thinking addresses would be 8 bits with 0xFF as a broadcast address, so datagrams sent to 0xFF are received by all devices.

The switch could have UARTs on it that function like a normal UARTs, and they could be presented on the network as streams. So one possible use could be expanding your UARTs, or sharing UARTs.

You could also bridge an RS232 from the network to a UDP port on a PC, then have a 'virtual switch' running in software using UDP. Brain melts.

Surely this has been done before? Very interested to hear thoughts.
 
purpose? are you referring to a multi node or perhaps multi master node network for ethernet usage?
if so then yes it can be done over a network, ive done one for CAN.
 
Purpose is simply to enable a bunch of devices to talk to each other, with built in layers of abstraction that facilitate real-time messaging through datagrams, or reliable transfers through streams, and separation of services or types of data through ports. The general concept I suppose puts it somewhere between CAN and ethernet in both speed and complexity.

The specific use case I have in mind is a system where one device (an autopilot) talks to a radio via RS232. There's a payload on the aircraft that I want to control via RS232, but the autopilot doesn't support that payload and I don't want to add another radio just for payload control. So I can't use it.
If the radio, autopilot and payload all used this hypothetical network for coms (or had interfaces hacked up for it), it'd be a total non-issue because the radio can talk to the payload AND the autopilot no worries at all. But you could ramp up the complexity from there. Throw a GPS or servo control onto the network. Have all your data spewing onto the network and throw a datalogger on there so when something dies you can try and debug it. (that'd only work if you broadcast everything so that datalogger would receive it). Similar to CAN except you can send data to a specific device whereas CAN sends everything to everyone because it's a bus.

Yeah it's a multi-node network with star like topology. Everything is a master.
 
Yeah it’s doable, when I wrote CANquitto it allowed every peripheral port access (SPI,I2C,UART,TOUCH,GPIO, ANALOG, etc) on any teensy to be controlled on the CAN network including transfers of 8K payloads, and any node can control each other even at the same time. Same could be done for ethernet if someone put time into it. Whats good about this is there is node detection and centralized code for a drop-in type of network

im planning to make an CANFD version in future
 
Last edited:
Alright, I'm doing this while I've got some holidays. Brief description: it's a LAN kind of like your home network, except it runs on Arduino/Teensy and uses UART. Going with 'picolan' for the project name.

For testing, I've got this pretend usart driver class that looks and feels like the Arduino serial class but actually uses UDP sockets. Using that, I've hacked up a switch, an echo server and sender to write/read from the echo server. So there's a bit of software pretending to be the switch and two other bits of software pretending to be devices talking to each other on the 'picolan'.

Here's the echo server

Code:
#include <iostream>
using namespace std;

#include "../ulan/cbus.h"


int main()
{
	Usart3.begin(1000000);
	ulan::Interface iface(Usart3);

	iface.set_address(9); // address is just an 8bit uint, valid range is 0-254 ( 255 is for broadcast )

	// create a socket and listen on port 22
        // ports are also 8bit unlike ip which is 16bit, so valid range is 0-255
	uint8_t buffer[1024];
	ulan::Socket socket(buffer, 1024, 22);
	iface.bind_socket(&socket);
	socket.set_timeout(0); // zero timeout for low latency replies

	uint8_t buf[1024];

	while(true) {
		// read up to 1kb of data
		auto len = socket.read(buf, 1024);
		auto remote = socket.get_remote();
		socket.write(remote, buf, len);
	}
}

And here's the sender
Code:
#include <iostream>
using namespace std;

#include "../ulan/ulan.h"

int main()
{
	Usart1.begin(1000000);
	// create ulan interface and attach it to the uart
	ulan::Interface iface(Usart1);

	// set the address for this device
	// address can be in the range 0-244
	// but the network can only handle 32 devices max
	iface.set_address(10);
	
	// create some buffer space for the socket
	// sockets use a fifo buffer to store received characters
	// if the buffer is too small you risk dropping packets
	// too big and you're wasting ram
	uint8_t buffer[128];

	// create a socket and listen on port 22
	// ports are 8bit, not 16 bit like IP networks
	// so valid range is 0-255
	ulan::Socket socket(buffer, 128, 22);


	iface.bind_socket(&socket); // bind socket to interface
	socket.set_timeout(10);  // 10ms timeout for reads

	// make a little buffer for reading/writing with the socket
	uint8_t b[20];
	uint32_t count = 0;
	while(true) {
		// use an etk::Rope to convert the count into text in the buffer
		etk::Rope rope((char*)b, 20);
		rope << count;

		socket.write(9, b, rope.length()); // write the text

		etk::set_array(b, 0, 20); // clear the buffer

		// read from the socket
		auto len = socket.read(b, 20);
		
		if(rope.atoi() != count) {
			cout << "found a discrepancy " << (char*)b << endl;
		}

		count++;
	}
}
That's example of usage of a socket, which is for sending/receiving data grams. I haven't worked out how I'm ganna implement a form of reliable messaging yet.

This is actual far more complex than I had imagined. So many little corner cases.
 
Screenshot from 2018-12-27 13-01-44.jpg

Builds for Arduino Nano & most likely will for Teensy but I've been getting some weird errors with compiling anything for Teensy so need to look into that . .
Unfortunately for UNO / Nano just this basic example uses 54% of memory. There's two reasons for this. 1# because I just cooked it up using existing code, zero effort put into optimisations yet. 2# all memory is preallocated and makes some assumptions about the size and quantity of data to be sent. That preallocated memory could be tuned.

Pings work too, for checking latency or whether a device is actually available at all. Latency times will depend on the ability of the switch to actually shuttle packets quickly, and on the devices themselves and how often the interface is serviced. The interface is serviced whenever you use socket read/write functions, or any other read/write function like ping, or by actually calling interface.read(), interface.serialise() and interface.flush().
Code:
#include <picolan.h>

picolan::Interface iface(Serial1);
  
void setup() {
  Serial.begin(57600);
  Serial1.begin(1000000);
  iface.set_address(9);
}

void loop() {
  uint32_t ping_time = 0;
    if(iface.ping(10, 1000, ping_time) == picolan::ERROR_NONE) {
      // ping_time contains the number of milliseconds it took
      // for device '10' to respond to the ping
      Serial.print(ping_time);
      Serial.print('\n');
    }
}
 
is there any data integrity checking when dealing with UDP (missing frames?), or perhaps you will implement both a TCP and UDP protocol based on the messanging data?
 
There is no checking for missing frames with datagrams. The socket implementation just blasts out data and hopes for the best. There is error checking at the frame level, but it's just check sums so if there's a corrupt frame it gets dropped. I'm not really sure what the best approach is for a reliable protocol. I don't really wanna do a full TCP-like thing with client, server, multiple connection states and flow control. It seems very complex and a little excessive for something with 'pico' in the title. It could be done very simply by just waiting for an ACK for every frame, but then you sacrifice some throughput. I dunno, still mulling over it.
 
when i attempted a UDP like protocol transfer along time ago, i thought about resending but also making sure a duplicate was not sent and processed twice, it can be tricky but there was a timeout after a 2-3 way talk transfer, before the endpoint would process the message, anything that happened in-between would be discarded and the timeout would resend a new frame with a new packet ID to be identified as a new transfer

pretty redundant but indeed complex at times, UDP is more of a fire-and-forget protocol but is very good at speed as it doesn’t have the overhead of TCP, and on a local network can pickup other nodes in a single broadcast. interesting work
 
Alright, well, I've got something working for reliable transfers. It is client/server based with multiple connection states . . . :rolleyes: Sort of superficially TCP-like but lacks flow control and many of the other features of TCP. The transfers work by sending bursts of up to 8 frames at a time (around 1600 bytes). Each frame has a sequence byte which is incremented per frame sent. When the receiver reads a frame, it checks that the sequence byte is exactly +1 from the last sequence byte. If so it will ack the new frame, if not it will ack the last frame. So if a frame is dropped, the receiver will just keep acking the last in-order frame. The sender, after sending up to 8 frames, will wait (for up to 50ms) to see if receives an ack for the last frame sent. Then it will go back to the last acked frame and do another burst of up to 8 frames. If no frames are dropped, I'm predicting maybe 80kB/s transfer speeds vs 86kB/s for datagrams at 1Mb/s baud rate. Gut feeling is there's plenty of room for improvement, but that's already fast enough for 44.1kHz 16-bit single channel audio.

Echo server
Code:
#include <iostream>
using namespace std;

#include "../picolan/picolan.h"

int main()
{
	Usart1.begin(1000000);
	picolan::Interface iface(Usart1);

	iface.set_address(10);

	uint8_t sock_buffer[128];
	picolan::Server server(sock_buffer, 128, 22);
	server.set_timeout(10);
	iface.bind(&server);

	while(true) {
		server.listen();
		while(server.closed() != true) {
			if(server.connection_pending()) {
				auto res = server.accept();
				if(res == picolan::ERROR_NONE) {
					uint8_t msg_buf[128];
					etk::set_array(msg_buf, 0, 128);
					while(server.connected()) {
						auto len = server.read(msg_buf, 128);
						if(server.write(msg_buf, len) == picolan::ERROR_TIMEOUT) {
							server.disconnect();
						}
					}
					server.disconnect();
					cout << "disconnected" << endl;
				} else {
					cout <<  "error accepting" << res << endl;
				}
			}
			iface.read();
			iface.serialise();
		}
	}
}

Client test/example

Code:
#include <iostream>
using namespace std;

#include "../picolan/picolan.h"

int main()
{
	Usart2.begin(1000000);
	picolan::Interface iface(Usart2);

	iface.set_address(9);

	uint8_t sock_buffer[128];
	picolan::Client client(sock_buffer, 128, 25);
	client.set_timeout(100);
	iface.bind(&client);

	while(true) {
		uint32_t ping_time;
		if(iface.ping(10, 1000, ping_time) == picolan::ERROR_NONE) {
			cout << "ping time: " << ping_time << endl;
			cout << "connecting . . ." << endl;
			auto res = client.connect(10, 22);
			if(res  == picolan::ERROR_NONE) {
				uint8_t buf[20];
				etk::Rope rope((char*)buf, 20);
				rope.clear();
				rope << "Hello world!";

				client.write(buf, rope.length());

				rope.clear();

				client.read(buf, 20);

				if(rope == "Hello world!") {
					cout << "great success" << endl;
				}
				client.disconnect();
			} else {
				cout << "error connecting: " << res << endl;

			}
		}
		sleep_ms(1000);
	}
}
 
Screenshot from 2018-12-30 11-43-51.jpg
Looking a bit better, so should run on UNO/Nano no worries. Walk in the park for Teensy 3.x.
6 port switch design sent off to osh park. Woot!
 
Just a little update

I got the PCBs from oshpark and assembled them, discovered mistakes and had to do microsurgery to reroute traces using strands of wire. The switch has 6 ports and is using DMA for writing and interrupts for continuous reads so should be pretty performant. Right now I've got a Teensy 3.2 broadcasting and my PC plugged in to another switch port using USB/serial, and I'm receiving the broadcast at my pc. Great success. :cool:

The problem I've got now is conceptual rather than technical, and I'm hoping the clever minds here can advise. The picolan library as is right now has datagrams and socket streams which are broadly similar to UDP and TCP respectively, except there is no equivalent to UDP multicast (yet). I'm struggling to wrangle my particular use cases into those categories. I need to have a multicast-like function so that a GPS, for instance, could transmit to many devices at once without spamming the network with a broadcast. I have a low bandwidth radio that will not tolerate the traffic if everything is broadcast.

Rather than multi-cast, I was thinking perhaps of borrowing concepts from DDS where there are 'topics' and topics have publishers and subscribers. The GPS would create a 'gps' topic and publish data. Devices that want to receive GPS data would subscribe to the 'gps' topic. The network would be responsible for tracking topics and sending data from publishers to all the relevant subscribers. It's pretty much multicast except instead of using port numbers, an ASCII string is used to subscribe and differentiate services and types of data.

Any thoughts or advice appreciated.
 
Status
Not open for further replies.
Back
Top