(Serial) messaging library for Teensy and .NET/Mono, with thermostat example

[thijse]

Hi,

A few months ago I wanted my Arduino to communicate with a C# application in order to log some data. I had expected to find a messaging library without any issue. However I could only find one library that came close: It implemented
- Both sending and receiving benchmarks
- sending multiple arguments per command command
- Trigger specific callback functions on received commands

But it was still lacking in some aspects. I updated the library so that:
- It can send all data-primitives (char arrays, floats, ints, longs, bytes) instead of just strings.
- It can wait for an acknowledge command after sending
- It has the ability to escape data. Special characters that would otherwise be interpreted as field separators or command separators are can be escaped and are made harmless.
- It has the ability to send binary data. Using escaping, basic data-types can be send over in binary form, for efficiency and accuracy

However, most work went into writing an implementation of the library from scratch in C# that runs both on .NET and Mono. In the time after the initial release I improved the library to work both at low speeds (i.e. a physical serial port) and high speeds (i.e. a virtual port running at full USB speed) by implementing my own serial port communication, that adapts dynamically to transfer tates. This means that it will hopefully run quite fast on a Teensy 3.

I quickly tried it out on a Teensy 3 of a friend, and at first glance it seems to work smoothly. Unfortunately I had little time to play with it (I was amazed at how small it is!) and was not yet able to do any benchmarks.
So I would like to ask everybody who is interested to try out the library with the Teensy and let me know how it works. Maybe somebody would even run some speed benchmarks?

In the latest version I also added an hopefully useful sample of a PC controlled, Arduino based thermostat. It uses a K-type thermocouple and a MAX31855 / Max6675 breakout board to measure temperature, a Solid State Relay to turn a heater on and off, and a PID controller implementation to steer the temperature optimally (no overshoots) to the goal temperature. My personal aim is to update our Gaggia Classic espresso machine with temperature stabilization, but this is sample set up to be generic enough that you can use it for anything: brewing beer, controlling a clay oven, etc.



Anyway, my rather expansive posts on the library on the Arduino forum can be found here:
forum.arduino.cc/index.php?topic=186928

And a even more detailed explanation can be found on my blog, together with a bunch of examples:
http://thijs.elenbaas.net/2013/09/ar...-pc-messaging/
and
http://thijs.elenbaas.net/2013/11/ar...tocols-speeds/

You can download the library here:
http://thijs.elenbaas.net/downloads/?did=9

Or at the Github repository:
https://github.com/thijse/Arduino-Li...r/CmdMessenger

If anybody want to try the library, please let me know what your experiences are.

Cheers, Thijs

[PaulStoffregen]

I'm looking at the code now.

The first issue is #include "Streaming.h". On AVR, missing headers are only warnings, but on ARM they're errors. You really need to delete that line, or put a copy of Streaming.h in your library.

In terms of performance, the main issue I see is processing input 1 byte at a time. That nice and simple, but slow. The Stream class has a readBytes() function that reads many bytes at once. On Teensy 2.0 and 3.0, readBytes() is optimized. Here's some benchmarks to show you want an incredible difference it makes:

http://www.pjrc.com/teensy/benchmark...l_receive.html

On Teensy 3.0, readBytes() is capable of receiving at the full USB speed AND leaving half the CPU time available. On normal Arduino boards, readBytes() defaults to nearly the same as reading the data 1 byte at a time, but odds are good they'll someday copy my optimization (usually stuff I do tends to filter into mainstream Arduino in about 2 years).

Your feedinSerialData() would be much faster if you integrated the processAndCallBack() and processLine() code into it. Instead of calling 2 functions for each byte (after having called 2 virtual functions merely to get each byte), you could make one call to readBytes() to fill a small buffer. Inside processLine(), it seems like you're putting the data into a buffer anyway. But this involves so much overhead to track state. The more of that state stuff you can put directly into feedinSerialData() as local variables (which are allocated into fast CPU registers), the faster your code will run. Even just eliminating the 4 function calls per byte will really speed this reception.


On the PC side, I mainly use Linux. I've never used Mono, so I'm not familiar with how to run your C# code. Any suggestions?


About the library itself... I have long hoped someone would develop a really awesome data sharing library, leveraging C++ operator overloading and a protocol that auto-negotiates types, sizes and really releaves the user from worrying about almost all details. I've considered doing this myself someday. In fact, I did some initial work on this with the Flight Sim Control USB type for X-Plane. I'm not sure if you want to take this library in that direction? But if you did, I'd really like to help as I can.

[thijse]

Thanks for the feedback!

I thougth I had removed the Streaming.h references. They are not needed, I just use the streaming lib to easily include debug statements. I will look into using readBytes(), I don't think it will be too hard to implement that. The benchmarking charts are very interesting, I can't wait to compare my Arduino ATmega with the Teensy ARM !

I have found the verbose compiler switch in the Arduino IDE and it shows multiple warnings. I aim currently fixing these.

I have no experience in developing C# under Linux, but I know that there are ports of MonoDevelop for different distro's, see http://monodevelop.com/Download. Under windows, if I download MonoDevelop I get redirected to XamarinStudio, which is more or less the commercial MonoDevelop with IOS / Android support. It is free and works quite well. I assume that MonoDevelop works similarly. When I have some time, I will set up a Linux box and try it out.

[thijse]

I just published an updated version of CmdMessenger, now at release 3.6. One of the mayor changes is the Teensy performance optimization. I started with seeing if I could reproduce Paul's benchmarks. Indeed, my Teensy 3 (the starred entry) performed similarly:


Paul did not benchmark the Arduino Uno or Arduino Nano. On Both boards the serial baudrate is limited to 115200 baud, but I was still curious to see how that translates out in the benchmarks. I added them because, while they are architecturally very different to the Teensy, they are similar in price. As expected, the transfer speeds are quite different: both 16 bit boards are capable of receiving +/-11500 bytes/sec, which is reasonably close to 115200 bits/sec / (8 bits + 1 stopbit), whereas the Teensy comes close to 1 MB/sec.

An interesting find by Paul was that Windows 7 performs much worse than Linux and OS-X for small data-blocks. I was curious to see if that had changed with Windows 7.1 and Windows 8. As you can see in the chart below, indeed it has, but not all for the better:


The Windows 7.1 results are slightly different, but that can be due to the hardware (HP 8560w: i7 2820QM, 2.3 GHz ) or compilers (PC: gcc-4.7.1, Arduino: Arduino 1.0.5) used. Under the same conditions Windows 8 performs better than Windows 7 for smaller data-blocks, although still not nearly as good as Linux and Mac. Strangely enough, though, for larger data-chunks, Windows 8 performs significantly worse than Windows 7.1!

This served all as a prelude to measuring the speed of CmdMessenger. I had already switched from using readByte() to readBytes(), but my first tests where pretty disappointing: the receiving speed was only twice that of an the Arduino Uno. It occurred to that while the Teensy was now able to receive and process many bytes at once, the PC side of CmdMessenger still send only very small messages, typically around 10 bytes. which

However, because the PC CmdMessenger has a nice send queue for asynchronously sending commands, I could combine multiple messages on the queue. This results in a single long string of multiple commands, which makes sending it much more efficient. As it turns out, this makes a world of difference, as you can see in the chart below. Compare the buffered and unbuffered send benchmark.


Obviously, it is still not as fast as sending raw data, but it is now 30 times faster than the Arduino Uno / Nano. I think that is a pretty excellent result!

Anyway, you can download it and read more about this release here

[Frank B]

The difference in speed is not very surprising, since the nano has an serial<>usb converter.

[thijse]

Indeed, the Uno and Nano are quite close to their maximum baudrate of 115200. The Teensy was also performing as expected. But as the PJRC benchmark page suggests: "We strongly encourage you to embrace the Soviet-era doctrine: Trust, but Verify."

[michelesmarty]

Hi,

A few months ago I wanted my Arduino to communicate with a C# application in order to log some data. I had expected to find a messaging library without any issue. However I could only find one library that came close: It implemented
- Both sending and receiving benchmarks
- sending multiple arguments per command command
- Trigger specific callback functions on received commands

But it was still lacking in some aspects. I updated the library so that:
- It can send all data-primitives (char arrays, floats, ints, longs, bytes) instead of just strings.
- It can wait for an acknowledge command after sending
- It has the ability to escape data. Special characters that would otherwise be interpreted as field separators or command separators are can be escaped and are made harmless.
- It has the ability to send binary data. Using escaping, basic data-types can be send over in binary form, for efficiency and accuracy

However, most work went into writing an implementation of the library from scratch in C# that runs both on .NET and Mono. In the time after the initial release I improved the library to work both at low speeds (i.e. a physical serial port) and high speeds (i.e. a virtual port running at full USB speed) by implementing my own serial port communication, that adapts dynamically to transfer tates. This means that it will hopefully run quite fast on a Teensy 3.

I quickly tried it out on a Teensy 3 of a friend, and at first glance it seems to work smoothly. Unfortunately I had little time to play with it (I was amazed at how small it is!) and was not yet able to do any benchmarks.
So I would like to ask everybody who is interested to try out the library with the Teensy and let me know how it works. Maybe somebody would even run some speed benchmarks?

In the latest version I also added an hopefully useful sample of a PC controlled, Arduino based thermostat. It uses a K-type thermocouple and a MAX31855 / Max6675 breakout board to measure temperature, a Solid State Relay to turn a heater on and off, and a PID controller implementation to steer the temperature optimally (no overshoots) to the goal temperature. My personal aim is to update our Gaggia Classic best espresso machine reviews 2017, but this is sample set up to be generic enough that you can use it for anything: brewing beer, controlling a clay oven, etc.



Anyway, my rather expansive posts on the library on the Arduino forum can be found here:
forum.arduino.cc/index.php?topic=186928

And a even more detailed explanation can be found on my blog, together with a bunch of examples:
http://thijs.elenbaas.net/2013/09/ar...-pc-messaging/
and
http://thijs.elenbaas.net/2013/11/ar...tocols-speeds/

You can download the library here:
http://thijs.elenbaas.net/downloads/?did=9

Or at the Github repository:
https://github.com/thijse/Arduino-Li...r/CmdMessenger

If anybody want to try the library, please let me know what your experiences are.

Cheers, Thijs

I really happy to read the post you have shared with us. You have provided a best information. Thanks