Augmenting Teensy millis timing with external RTC

[mnissov]

I'm using a teensy 4.1 for synchronizing and time-stamping of sensors, as such consistency of the timestamping is important.

I bought an RV3028 RTC (1ppm, https://shop.pimoroni.com/products/r...27926940549203) with the goal of more accurate timestamping, without realizing it provides stamp down to the second. My output signals will be something on the order of 20-200 Hz, therefore timestamp precision in the order of sub- to 1-millisecond precision is necessary. How should I solve this problem?

One potential solution I can think of is using the 32.768 kHz clkout signal to increment a counter and use this to augment the stamping. This approach should have a resolution of 0.030517578 ms, but I'm worried that it might be taxing for the system to have to read and increment a variable at this rate. Of course I could use a lower clkout signal rate (e.g. 8192 Hz -> 0.122070313 ms resolution) but this also seems like an imperfect solution for the same reasons

Is it possible to synchronize the `millis` on the teensy with the RTC? Is there a better approach which I'm not considering?

[luni]

Actually 32kHz is not very fast for a T4.1. You can connect the clkout signal to a Teensy pin and attach a pin interrupt. In the ISR I'd simply increment a 32bit variable and use this as timestamp.
Something like (untested):

constexpr int clkPin = 42;  // some pin
volatile uint32_t currentCount = 0;  // volatile since it is changed in an interrupt routine

void onClock()
{
    currentCount++;
}

void setup()
{
    pinMode(clkPin, INPUT_PULLUP);
    attachInterrupt(clkPin, onClock, RISING);
}

void loop()
{
    Serial.println(currentCount);  // current count incremented every 1/32768 sec. 
    delay(1000);
}

[rcarr]

You could use an elapsed micros variable. You would reset the variable to zero on each change of the second on your external clock.

https://www.pjrc.com/teensy/td_timin...pedMillis.html

[mnissov]

Actually 32kHz is not very fast for a T4.1. You can connect the clkout signal to a Teensy pin and attach a pin interrupt. In the ISR I'd simply increment a 32bit variable and use this as timestamp.
Something like (untested):

constexpr int clkPin = 42;  // some pin
volatile uint32_t currentCount = 0;  // volatile since it is changed in an interrupt routine

void onClock()
{
    currentCount++;
}

void setup()
{
    pinMode(clkPin, INPUT_PULLUP);
    attachInterrupt(clkPin, onClock, RISING);
}

void loop()
{
    Serial.println(currentCount);  // current count incremented every 1/32768 sec. 
    delay(1000);
}

I'll look into something like this, a bigger problem I just noticed is my breakout doesn't actually expose the CLKOUT pulse pin

[mnissov]

You could use an elapsed micros variable. You would reset the variable to zero on each change of the second on your external clock.

https://www.pjrc.com/teensy/td_timin...pedMillis.html

This will still be based on whatever timing mechanism there exists in the teensy 4.1, with no way of utilizing the external RTC. So I'm not sure this really fits the purpose.

[BriComp]

You have the INTerrupt pin on the RV3028. This can be set up for repetitive interrupt. See the RV-3028 Manual.

[rcarr]

This will still be based on whatever timing mechanism there exists in the teensy 4.1, with no way of utilizing the external RTC. So I'm not sure this really fits the purpose.

I would see this working like this: you set the RV-3028 to generate a 1 second interrupt. In servicing the interrupt you read the time via I2C and reset your elapsed micros variable to zero. You now have your timestamp information locally to the Teensy and will not need to read the real time clock again until the next second. Caveats : the wire library is probably not interrupt safe, so you may need to set a flag and read the real time from loop().

https://www.microcrystal.com/fileadm...App-Manual.pdf

[Mcu32]

In servicing the interrupt you read the time via I2C and reset your elapsed micros variable to zero

Oh.. not a good Idea. Never use a really slow protocol like I2C in a ISR. This will always slow down your program exactly when you don't want it to.
Don't do it like that.

Edit: If you read the time via i2c (regardless where - inside a ISR or loop) and want it exact, you may want to take the time it takes to read the time into account.

[BriComp]

The following might give you some ideas of what is possible.
Note I have compiled it but NOT RUN/TESTED it!

// Visual Micro is in vMicro>General>Tutorial Mode
// 
/*
    Name:       t4.ino
    Created:	14/12/2022 19:50:52
    Author:     
*/

#include <RV-3028-C7.h>

RV3028 rtc;

typedef struct timeStampType {
    uint32_t unixSeconds;
    uint32_t microSeconds;
};

timeStampType timeStamp;

uint32_t      unixTime;
elapsedMillis secondsX1000;
elapsedMicros microSeconds;

constexpr int clkPin = 42;  // some pin

volatile uint32_t currentCount = 0;  // volatile since it is changed in an interrupt routine

void onClock()
{
    microSeconds = 0;
}

void setup()
{
    Serial.begin(9);
    while (!Serial && millis() < 5000);
   
    Wire.begin();
    if (rtc.begin() == false) {
        Serial.println("Something went wrong, check wiring");
        while (1);
    }
    else
        Serial.println("RTC online!");

    pinMode(clkPin, INPUT_PULLUP);
    attachInterrupt(clkPin, onClock, RISING);

    rtc.enablePeriodicUpdateInterrupt(true, true);
    unixTime     = rtc.getUNIX();
    secondsX1000 = 0; // Not really necessary
}

timeStampType GetTimeStamp() {
    timeStampType ts;

    ts.microSeconds = microSeconds;
    ts.unixSeconds  = unixTime + (secondsX1000 / 1000);
    return ts;
}
bool somethingHappened;

// Add the main program code into the continuous loop() function
void loop()
{
    if (somethingHappened)
        timeStamp = GetTimeStamp();
}

[defragster]

The @rcarr solution seems good if a single elapsedMicros value could be used and maintained for reference. Perhaps kept with an added count of 'elapsed seconds'. With a 32 bit count of second that would cover 136 years.

I've measured some T_3.6 and T_4.0 against a GPS PPS and watching the cycle counter they all do fewer cycles per second than clock speed would suggest.

Just happen to have two T_4.0's running now - each against a different GPS and after some DAYS they are showing:

us 3458412057 cyc diff 131 err= 0.22 us [4978 cyc diff Mn=4636 Mx=6576 P= 999991.75 ( 999991.69

and

us 2809640796 cyc diff 124 err= 0.21 us [1598 cyc diff Mn=24 Mx=1910 P= 999997.25 ( 999997.25

So 600Mhz clock is showing "Mn=4636 Mx=6576" cycles short on the one and "Mn=24 Mx=1910" on the other.

That is the range of cycles counts seen short of the expected 600,000,000 cycles when the GPS PPS reported the PPS second interrupt.

IIRC results some years back on T_3.6 were similar, one PPS second always elapsed before the processor cycle counter completed 'one second' of clock rate cycles.

This code has some sort of PID math to correct the cycle count expected per reference second from the PPS. On one it needs an IntervalTimer around 999991.69 us to match the PPS second and on the other 999997.25 us.

As temp and processing change through the day with the units in the window for GPS sky view the crystal cycles change - but ideally the GPS PPS signals are as expected. This would be similar to the results form a more accurate PPS from the RV-3028 giving one second setpoints.

Once the RV-3028 is running and the PPS triggering the use case may not need to ever read the i2c time value if 32bits each of "seconds.microseconds" is good enough for event time stamping. But with that change to PPS triggered by the RV-3028 the time would stay in sync, that is every 3,600 seconds would be an hour as counted by the RV-3028.

[defragster]

The following might give you some ideas of what is possible.
...

volatile uint32_t      secondsCnt = 0;
elapsedMicros microSeconds;

// ...

void onClock()
{
    secondsCnt++;
    microSeconds = 0;
}
// ...

Code above looks like a good base toward the @rcarr idea of syncing the elapsedMicros - p#10 suggested a local second count that would look like in edit here

[mnissov]

Oh.. not a good Idea. Never use a really slow protocol like I2C in a ISR. This will always slow down your program exactly when you don't want it to.
Don't do it like that.

Edit: If you read the time via i2c (regardless where - inside a ISR or loop) and want it exact, you may want to take the time it takes to read the time into account.

As a combination of some of the aforementioned ideas, perhaps I can rely on `micros` locally, and align to the second signal from the RTC. Instead of doing it through I2C I can get a 1Hz interrupt pulse from the INT pin, as suggested by @BriComp and then reset the micros since last second counter in the interrupt. Perhaps something in this regard makes sense.


The alternative in my mind is just keeping a counter at the 244 us resolution the RTC INT can max provide and use this for timekeeping, but micros provides a nicer resolution without reducing the accuracy. At least I guess it shouldn't drift significantly over a second right?

[mnissov]

The @rcarr solution seems good if a single elapsedMicros value could be used and maintained for reference. Perhaps kept with an added count of 'elapsed seconds'. With a 32 bit count of second that would cover 136 years.

I've measured some T_3.6 and T_4.0 against a GPS PPS and watching the cycle counter they all do fewer cycles per second than clock speed would suggest.

Just happen to have two T_4.0's running now - each against a different GPS and after some DAYS they are showing:



and


So 600Mhz clock is showing "Mn=4636 Mx=6576" cycles short on the one and "Mn=24 Mx=1910" on the other.

That is the range of cycles counts seen short of the expected 600,000,000 cycles when the GPS PPS reported the PPS second interrupt.

IIRC results some years back on T_3.6 were similar, one PPS second always elapsed before the processor cycle counter completed 'one second' of clock rate cycles.

This code has some sort of PID math to correct the cycle count expected per reference second from the PPS. On one it needs an IntervalTimer around 999991.69 us to match the PPS second and on the other 999997.25 us.

As temp and processing change through the day with the units in the window for GPS sky view the crystal cycles change - but ideally the GPS PPS signals are as expected. This would be similar to the results form a more accurate PPS from the RV-3028 giving one second setpoints.

Once the RV-3028 is running and the PPS triggering the use case may not need to ever read the i2c time value if 32bits each of "seconds.microseconds" is good enough for event time stamping. But with that change to PPS triggered by the RV-3028 the time would stay in sync, that is every 3,600 seconds would be an hour as counted by the RV-3028.

With regards to this, I believe you're right. Note I won't have access to a GPS, but the duration that the RTC time needs to remain accurate is likely less than a week, so I believe you're right that 32 bits each of unxtime and microseconds since last second should be totally sufficient in accurate timestamping. At least as accurate as can be with the given tools.

Of course the triggering signals themselves are still originating from IntervalTimers, so I guess this sufferes from clock deviations in the teensy, but honestly as long as the frequency doesn't change drastically with accurate timestamping it shouldnt matter.

[mnissov]

The following might give you some ideas of what is possible.
Note I have compiled it but NOT RUN/TESTED it!

// Visual Micro is in vMicro>General>Tutorial Mode
// 
/*
    Name:       t4.ino
    Created:	14/12/2022 19:50:52
    Author:     
*/

#include <RV-3028-C7.h>

RV3028 rtc;

typedef struct timeStampType {
    uint32_t unixSeconds;
    uint32_t microSeconds;
};

timeStampType timeStamp;

uint32_t      unixTime;
elapsedMillis secondsX1000;
elapsedMicros microSeconds;

constexpr int clkPin = 42;  // some pin

volatile uint32_t currentCount = 0;  // volatile since it is changed in an interrupt routine

void onClock()
{
    microSeconds = 0;
}

void setup()
{
    Serial.begin(9);
    while (!Serial && millis() < 5000);
   
    Wire.begin();
    if (rtc.begin() == false) {
        Serial.println("Something went wrong, check wiring");
        while (1);
    }
    else
        Serial.println("RTC online!");

    pinMode(clkPin, INPUT_PULLUP);
    attachInterrupt(clkPin, onClock, RISING);

    rtc.enablePeriodicUpdateInterrupt(true, true);
    unixTime     = rtc.getUNIX();
    secondsX1000 = 0; // Not really necessary
}

timeStampType GetTimeStamp() {
    timeStampType ts;

    ts.microSeconds = microSeconds;
    ts.unixSeconds  = unixTime + (secondsX1000 / 1000);
    return ts;
}
bool somethingHappened;

// Add the main program code into the continuous loop() function
void loop()
{
    if (somethingHappened)
        timeStamp = GetTimeStamp();
}

I've seen a number of libs on github for this RTC. Not to be mean, but of varying quality. Which one is

RV-3028-C7.h

referring to? I've seen a couple with this name on github.

[luni]

Is there a reason why you can't use the internal RTC for generating the timestamps and the triggering signals? What are your accuracy requirements?

[mnissov]

Is there a reason why you can't use the internal RTC for generating the timestamps and the triggering signals?

As farr as I know the internal RTC only allows interrupts at a single frequency and we need at least 2, so figured interval timers would be more straightforward. Also assumed using interrupts would be "cleaner" than checking for

ellapsedMillis - previousMillis > trigger_period

in the main loop. Regarding for stamping, I think it's 20ppm which is a little worse than we hoped for.

What are your accuracy requirements?

This doesn't have an extremely explicit answer at the moment, but we're just trying to go as accurate as possible for a reasonable amount of effort in order to support future endeavors which may require more or less accuracy. I would say generally sub 5ms drift over 10 mins could be something of an upper limit. If I'm not mistaken this, roughly, corresponds to 10ppm. The 1ppm RTC is cheap and simple enough that we figured this is a reasonable length to go.


In conclusion, the name of the game for us is mostly just to create as good of a system as possible given reasonable amounts of effort such that this doesn't become a bottleneck for some future activities, if that makes sense.

[BriComp]

I've seen a number of libs on github for this RTC. Not to be mean, but of varying quality. Which one is

RV-3028-C7.h

referring to? I've seen a couple with this name on github.

This is the one I have been using and is the one refered to in my post.
Obviously you have to be aware of rollover of the millis and micro, especially the latter. Perhaps do some zeroing/resetting each time you take a timestamp.

[luni]

IIRC the accuracy of the 32kHz crystal is about 20ppm. If that is still acceptable, you can try the TimerTool. Besides other Hardware timers it also supports the RTC as timing source.

E.g. the following code generates a 20ms pulse every second. Timing is based on the internal RTC. You can use up to 20 of those timers. Either in periodic or oneShot mode.

#include "Arduino.h"
#include "TeensyTimerTool.h"
using namespace TeensyTimerTool;

OneShotTimer t1(TCK_RTC);

void onT1()
{
    digitalWriteFast(0, LOW);
}

void setup()
{
    pinMode(0, OUTPUT);
    t1.begin(onT1);
}

void loop()
{
    digitalWriteFast(0, HIGH);
    t1.trigger(20ms);

    delay(1000);
}

Chaining a few oneShot timers (triggering the next timer from within a callback) is a simple way to generate timing patterns.

Using the RTC directly:
If you want to read out the RTC with full precision you can use the code from here: https://github.com/TeensyUser/doc/wi...eal-time-clock

[mnissov]

IIRC the accuracy of the 32kHz crystal is about 20ppm. If that is still acceptable, you can try the TimerTool. Besides other Hardware timers it also supports the RTC as timing source.

E.g. the following code generates a 20ms pulse every second. Timing is based on the internal RTC. You can use up to 20 of those timers. Either in periodic or oneShot mode.

#include "Arduino.h"
#include "TeensyTimerTool.h"
using namespace TeensyTimerTool;

OneShotTimer t1(TCK_RTC);

void onT1()
{
    digitalWriteFast(0, LOW);
}

void setup()
{
    pinMode(0, OUTPUT);
    t1.begin(onT1);
}

void loop()
{
    digitalWriteFast(0, HIGH);
    t1.trigger(20ms);

    delay(1000);
}

Chaining a few oneShot timers (triggering the next timer from within a callback) is a simple way to generate timing patterns.

Using the RTC directly:
If you want to read out the RTC with full precision you can use the code from here: https://github.com/TeensyUser/doc/wi...eal-time-clock

I do use teensy timer tool and the inbuilt IntervalTimers for generating the signals themselves. But for timestamping 20ppm is beyond what we want for accuracy (as I highlight in 16).


Just to be clear, I don't believe the RTC can create interrupts for more than one frequency. This is why I'm using interval timers, because I need at least 2 (maybe 3) frequencies, and there is no guarantee that they are multiples.

[mnissov]

This is the one I have been using and is the one refered to in my post.
Obviously you have to be aware of rollover of the millis and micro, especially the latter. Perhaps do some zeroing/resetting each time you take a timestamp.

Great, thanks! And yes of course, I think my current intention would be something along the lines of

set(){
    // initialize second_counter to RTC unixtime, i.e. seconds since 1 Jan 1970
    // ensures timestamp is, more or less, date accurate. Assuming RTC has been updated recently
}
ISR(PORT_NUMBER){
    micros_since_second = 0;
    second_counter++;
}
send_timestamp(){
    // code to send [second_counter, micros_since_second]
}

I hope this is sufficient to illustrate the idea from me, anyway thanks for the guidance.

[defragster]

@luni - i thought it was 20 as well - was wondering as the Sparkfun units I got claim ~2PPM - I found this note:

...
The clock stability will be based on the 24 MHz crystal Teensy 4.1 uses, which is a fairly ordinary 30 ppm, plus any "phase noise" from the PLL which multiplies it up to 600 MHz.

@mnissov - indeed GPS for PPS is accurate - but awkward and uncommon. Those two T_4.0's have been running another day or two since last post.

This one seems stable:

us 414435796 cyc diff 4294967153 err=-0.24 us [1324 cyc diff Mn=24 Mx=1328 P= 999997.75 ( 999997.81

The other catches some glitches? Not sure if the GPS misses PPS. It runs in a narrow window then the 'cycle Diff' count jumps:

 us 1139296550 cyc diff 4294966969 	err=-0.55 us	[4676 cyc diff Mn=2324 Mx=14924 P= 999992.06 ( 999992.12

They both run stable with drift through a day (Temp?) and as noted they seem to be short of cycles uniformly T_3.6 and these T_4.0's as measured against the GPS PPS. Depending on the unit they may be 1000 or 5000 cycles short per second.

I didn't power up the two learn.sparkfun.com/tutorials/qwiic-real-time-clock-module-rv-1805-hookup-guide units I just got yet. It seems it can enable an interrupt on the second so it can be substituted for a GPS at a claimed "±2.0 ppm"

The code here is crude and as noted attempts to modify the IntervalTimer on the fly to get it to match the GPS PPS report - above again about 2 and 8 us short of 1 GPS PPS second depending on the unit.

Those two T_4.0's look to be production (not beta) units - but were breadboarded to GPS before T_4.1 was released. And they don't refer to RTC time in any way - just tracking cycle count diff between IntervalTimer and GPS_PPS. It is possible the internal RTC unit gives diff results - but not monitored.

IIRC the T_3.6 crystal report could be adjusted per unit once it's behavior was catagorized? Possible the T_4.x's 1062 offers the same in a post somewhere?

[mnissov]

...
IIRC results some years back on T_3.6 were similar, one PPS second always elapsed before the processor cycle counter completed 'one second' of clock rate cycles.

This code has some sort of PID math to correct the cycle count expected per reference second from the PPS. On one it needs an IntervalTimer around 999991.69 us to match the PPS second and on the other 999997.25 us.
...

you mentioned some correction on the cycle count according to the PPS, how exactly do you do this? I see you mention an IntervalTimer, are you using this timer to increment the 1us counter? Mine also demonstrates the tendency of being behind by come CPU cycles when the 1s PPS arrives.

Perhaps it could make sense to multiply the micros by a scaling factor to make up for this difference? I visualize this as "stretching" the micros time base to fit the 1s duration. Having the IntervalTimer for this seems like a lot of processing to pay, maybe this is somehow for efficient?

Taking one of your examples, say we're 6576 cycles off, this corresponds to the factor (600e6 / (600e6 - 6576)) = 1.00001096. Such that multiplying (600e6 - 6576) cycles by the factor results in 600e6. My thought process is such that at a random time between two PPS signals t_cpu we can apply the approximate correction calculated at the last PPS interrupt such that t_est = t_cpu * correction_factor

Not sure if this makes sense