How to synchronize a gaggle of RS485 clients....

[Constantin]

Silly question of the evening (Merry Christmas!)

Let's say, you have a gaggle of RS485 clients on a half-duplex bus that is administered by one master unit (Teensy). The clients perform DAQ services that are cataloged on a second-by-second basis. Ideally, all measurements are concurrent, i.e. every second-long measurement cycle starts within 10ms or so. I have been thinking about the best way to synchronize the various MCUs attached to the bus and wonder if the following high-level approach makes sense:

1) Put a good clock signal on the master unit (i.e. SIT1552, for example)
2) The master notes how many us have elapsed since the last time that second() changed on the master. Store this offset.
3) Address all clients on the RS485 bus in a broadcast fashion and transfer the offset.
4) The clients use offset info to synchronize their measurement cycles to those of the master.
5) Every x hrs, re-synchronize the clocks.

This could be a good use for ElapsedMillis? I.e. base the timing of measurements using ElapsedMillis and then fix any offsets using the master unit.

 

[Constantin]

Thinking through this further, seems that one way to do it would be using the Metro timed interval and to reset it when appropriate. That is,
1) The master connects to a client and the client receives the offset signal (i.e. the elapsed time in ms relative to when the the master clock second() ticked over and the transmission commencing)
2) The client waits for the offset to elapse (i.e. 1000-offset), then resets the metrocounter, allowing the client to tick over in sync with the master clock.

Another good reason to use Metro is that it consumes 24 bytes less active memory. So if you don't need the extra functions, Metro is a better option for fixed interval operations. However, the reset function is particularly cool for helping large numbers of clients achieve somewhat synchronous measurements.

 

[PaulStoffregen]

RS485 broadcast probably makes the most sense. The tough problem with broadcast messaging is handling cases where some slave devices don't receive the message. But if they're designed to just keep using the previous time broadcast, maybe the worst case drift isn't so bad?

Another minor issue is designing your RS485 master to plan ahead for the time sync message. You'll want to stop transmitting new messages far enough in advance that any slave device you're expecting to reply can complete and the bus returns to idle before you need to broadcast the sync message. Using elapsedMillis or elapsedMicros can help you to plan your message timing.

This type of message-based sync (with good design) can achieve precision scale of a couple bit times at whatever baud rate you're using. Doing any better is really tough due to the asynchronous nature of serial.

 

[LenSamuelson]

I've been considering use of a timer/counter trigger input: In a design that I'm working on, I plan to have the master warn the clients of an upcoming synch, allowing them to prepare the timer/counters to trigger on the falling edge of the first start bit of the next broadcast message. That sync message contains a timestamp, which gives the clients known relationship between their clocks and the clock of the master without concern for interrupt service latency.

The project requires tight timing synchronization among a fairly large gaggle of sensors, and much of the focus of the comms design is to ensure that the timing of sensor readings can be computed accurately.

 

[Constantin]

I may have a first iteration of the code ready for tryouts. Now I have to reflow some teensy's to hook them all together.

FWIW, I presently plan on fixing the timing of every client after they have uploaded their latest payload of DAQ data. The idea being that each client uploads up to the last 60 seconds worth of data to the master, clearing their accumulators. The data is packaged in second-by-second data packages whose time is indicated as relative to the master clock. So a time stamp may be simply -15, indicating that this data package was recorded 15 seconds ago. Thus, timing offsets between clients can never be truly terrible, at most, shifted about a second apart.

But once all the data has been pulled out of the client, it's a good time to synchronize clocks. So the master sends a time update request. The master keeps track of micros() as the seconds() tick over. At the time of transmission, the master notes the difference between the micros() time when the second() changed and the present time in micros(). This is the offset from the master clock ticking over.

The client receives the offset and then adjusts the length of the present 'second' until it ends at the same time as the micro-second offset indicated by the master (i.e. wait 1,000,000 us - offset at time of transmission). That way, the system doesn't lose any data, some seconds will simply be longer or shorter than others. I plan on doing this maybe once an hour to help with any non-crystal units.

In an earlier iteration, I hooked up the output of a DS3231 to a TX-only configured RS485 interface to send accurate timing signals across a dedicated timing bus/cable pair. There are RX-only RS485 interfaces out there that can then interrupt a client MCU to the fact that another second ticked over, it's time to summarize data, etc. That approach is likely overkill, yet a good way to achieve a accurate bus timing signal without the serial data issues, interrupts affecting timing, and so on. There is the expense of the transmitter and the receiver plus the cable pair.. plus the required board space, decoupling capacitors and so on. But if you use RJ45-jacked ethernet cables for your RS485 bus (which I like because they're cheap, available pre-confectioned in many lengths, colors, and so on) then losing a cable pair is not necessarily a big deal.

My main reason for not implementing that approach anymore is that I'd like be able to use the same approach for timing synchronization with XBee communications. That is, use the same bus IDs for devices in the network and communicate with them over Xbee or RS485.