Teensy 4.1 RTC get milliseconds correctly

[klaasdc]

For a CAN logger project, I am trying to get a reliable timestamp from the RTC with millisecond precision. I have found several topics on this and worked out some code from that, but I get erratic behaviour in the resulting log files. Sometimes, seconds jump back and forth and the log is seemingly not chronological (even though the data says otherwise)

I have a function "rtc_get_ms" that is supposed to give me the actual second and millisecond, in conjunction with Teensyduino's minute(), hour() etc functions.
When writing a CAN message to the buffer, I store the time as such:

    //Store message and time
    uint32_t sec, ms;
    rtc_get_ms(&sec, &ms);
    lbuf[lbuf_head].second = second();
    lbuf[lbuf_head].minute = minute();
    lbuf[lbuf_head].hour = hour();
    lbuf[lbuf_head].day = day();
    lbuf[lbuf_head].month = month();
    lbuf[lbuf_head].year = year();
    lbuf[lbuf_head].millisecond = (unsigned int) ms;
    memcpy((void*) &lbuf[lbuf_head].msg, msg, sizeof(CAN_message_t));
    lbuf_head++;
    if (lbuf_head >= CANBUFSIZE){
        lbuf_head = 0;
    }

The millisecond rtc function is as follows:

void rtc_get_ms(uint32_t * seconds, uint32_t * milliseconds){
	uint32_t hi1 = SNVS_HPRTCMR;
	uint32_t lo1 = SNVS_HPRTCLR;
	while (1) {
		uint32_t hi2 = SNVS_HPRTCMR;
		uint32_t lo2 = SNVS_HPRTCLR;
		if (lo1 == lo2 && hi1 == hi2) {
      *seconds = (hi2 << 17) | (lo2 >> 15);
      *milliseconds = (lo2 & 0b00000000000000000111111111111111);
      *milliseconds = ((*milliseconds*1000)/32768);
      return;
		}
		hi1 = hi2;
		lo1 = lo2;
	}
}

What could be the cause of this 'jumping' of seconds? Is it the combination of teensyduino second() and rtc_get_ms()? Is there a proper way to read out the seconds and milliseconds "synchronously"?

Thanks for any advice,

 

[el_supremo]

I think the implementation of the rtc_get_ms function may be incorrect.
EDIT: Nope, my reading of that code was incorrect. Sorry.

Pete

 

[luni]

I can't help you with the ms readout of the RTC registers but, some months ago I investigated the possibilities of the new (c++11) std::chrono facility of c++. For fun I implemented a std::chrono compatible high precision Teensy_clock. It uses the cycle counter as time base which gives the clock a resolution of about 1.67ns (1/600MHz). Here some more info about this quest: https://github.com/TeensyUser/doc/wiki/fun-with-modern-cpp

So, if you have fun using modern c++ you could do the following:

#include "Arduino.h"
#include "teensy_clock.h"

using namespace std::chrono;
using timePoint = teensy_clock::time_point; // alias to save typing...

void setup()
{
    while(!Serial){}
    teensy_clock::begin(); // this will sync the teensy_clock to the RTC
}

void loop()
{
    constexpr uint32_t dt          = 110; // print timestamp every 'dt' microseconds
    static elapsedMicros stopwatch = dt;

    if (stopwatch > dt)
    {
        stopwatch -= dt;

        timePoint now = teensy_clock::now(); // get the current time from the teensy clock with a resolution of 1.67ns (1/600MHz)

        // currently (c++14) std::chrono  doesn't provide much support for transformation of chrono::timepoints to
        // years, day of month etc. c++20 would improve this a lot, but alas...
        // Meanwhile we can work around using the traditional time_t and struct tm to handle this

        time_t rawTime = teensy_clock::to_time_t(now); // convert the std::chrono time_point to a traditional time_t value
        tm t           = *gmtime(&rawTime);            // caluclate year, month... from the raw time_t value and store them in a tm struct
        int year       = t.tm_year + 1900;
        int month      = t.tm_mon + 1;
        int day        = t.tm_mday;
        int hour       = t.tm_hour;
        int minute     = t.tm_min;
        int second     = t.tm_sec;

        // since time_t is 'seconds since 1970-01-01', this of course can't handle smaller time units.
        // But we can use the teensy_clock directly for this:

        // e.g. microsecond resolution:
        int64_t usSinceEpoch = duration_cast<microseconds>(now.time_since_epoch()).count(); // total number of microseconds since 1970-01-01
        int us               = usSinceEpoch % 1'000'000;                                    // microseconds in the 'current second'
        Serial.printf("%d-%02d-%02d %02d:%02d:%02d %03.3f\n", year, month, day, hour, minute, second, us / 1000.0f);

        // millisecond resolution
        // int ms = duration_cast<milliseconds>(now.time_since_epoch()).count() % 1'000;
        // Serial.printf("%d-%02d-%02d %02d:%02d:%02d %03u\n", year, month, day, hour, minute, second, ms);
    }
}

The code prints out the current time every 110µs with microseconds resolution (the commented part would print with millisecond resolution).
Here the printout around a 'second switch'

....
2021-08-29 09:09:56 998.910
2021-08-29 09:09:56 999.020
2021-08-29 09:09:56 999.130
2021-08-29 09:09:56 999.240
2021-08-29 09:09:56 999.350
2021-08-29 09:09:56 999.460
2021-08-29 09:09:56 999.570
2021-08-29 09:09:56 999.680
2021-08-29 09:09:56 999.790
2021-08-29 09:09:56 999.900
2021-08-29 09:09:57 0.010
2021-08-29 09:09:57 0.120
2021-08-29 09:09:57 0.230
2021-08-29 09:09:57 0.340
2021-08-29 09:09:57 0.450
2021-08-29 09:09:57 0.560
...

Please note: Since the clock uses the cycle counter as time base it might (will) slowly drift away from the RTC. Probably not an issue for generation of time stamps but should be mentioned...

I never bothered making this a library. -> Sources attached (just copy them to the sketch folder)

 

[luni]

Addendum:
You can also directly use the RTC values to feed the predefined chrono::system_clock. To use the system_clock on a controller all you need to do is overriding the system_clock::now() function. The system_clock uses nanoseconds as time base, thus the now() function needs to return the nanoseconds since 1970-01-01. Since the RTC keeps the number of oscillator periods since this date in its in the SNVS_HPRTCMR and SNVS_HPRTCLR the conversion to nanoseconds is simple.

Here the same code as above using the RTC based system_clock instead of the cycle counter based teensy_clock. Please note that this is self contained, no additional code / libraries needed.

#include "Arduino.h"
#include <chrono>

using namespace std::chrono;

// get the number of oscillator periods since 1970-01-01
uint64_t get_RTC_periods()
{
    uint32_t hi1 = SNVS_HPRTCMR, lo1 = SNVS_HPRTCLR;
    while (true)
    {
        uint32_t hi2 = SNVS_HPRTCMR, lo2 = SNVS_HPRTCLR;
        if (lo1 == lo2 && hi1 == hi2)
        {
            return (uint64_t)hi2 << 32 | lo2;
        }
        hi1 = hi2;
        lo1 = lo2;
    }
}

// this is all you need to do to prepare the built in system_clock for usage with the RTC (resolution ~30.5µs)
system_clock::time_point system_clock::now()
{
    uint64_t ns = get_RTC_periods() * (1E9 / 32768); // system clock uses ns as time base (would be better to define dedicated clock with osc periods as base)
    return system_clock::time_point(system_clock::duration(ns));
}

using timePoint = system_clock::time_point; // alias to save typing...

//-----------------------------

void setup()
{
    while (!Serial) {}
}

void loop()
{
    constexpr uint32_t dt          = 100'000; // print timestamp every 'dt' microseconds
    static elapsedMicros stopwatch = dt;

    if (stopwatch > dt)
    {
        stopwatch -= dt;

        timePoint now = system_clock::now(); // get the current time from the system clock with a resolution of 30.5µs (1/32.768kHz)

        // currently (c++14) std::chrono  doesn't provide much support for transformation of chrono::timepoints to
        // years, day of month etc. c++20 would improve this a lot, but alas...
        // Meanwhile we can work around using the traditional time_t and struct tm to handle this

        time_t rawTime = system_clock::to_time_t(now); // convert the std::chrono time_point to a traditional time_t value
        tm t           = *gmtime(&rawTime);            // caluclate year, month... from the raw time_t value and store them in a tm struct
        int year       = t.tm_year + 1900;
        int month      = t.tm_mon + 1;
        int day        = t.tm_mday;
        int hour       = t.tm_hour;
        int minute     = t.tm_min;
        int second     = t.tm_sec;

        // since time_t is 'seconds since 1970-01-01', this of course can't handle smaller time units.
        // But we use the teensy_clock directly for this:

        int ms = duration_cast<milliseconds>(now.time_since_epoch()).count() % 1'000;
        Serial.printf("%d-%02d-%02d %02d:%02d:%02d %03u\n", year, month, day, hour, minute, second, ms);
    }
}

Here the printout for dt = 100ms

2021-08-29 11:39:38 295
2021-08-29 11:39:38 395
2021-08-29 11:39:38 495
2021-08-29 11:39:38 595
2021-08-29 11:39:38 695
2021-08-29 11:39:38 795
2021-08-29 11:39:38 895
2021-08-29 11:39:38 995
2021-08-29 11:39:39 095
2021-08-29 11:39:39 195
2021-08-29 11:39:39 295
2021-08-29 11:39:39 395
2021-08-29 11:39:39 495
2021-08-29 11:39:39 595
2021-08-29 11:39:39 695

 

[klaasdc]

Thanks Luni, that is definately useful for me!

At the moment I am seeing that the value from seconds() in the log file is stepping back and forth.
E.g. 13:09:37, 13:09:37, 13:09:38 ... , 13:09:37, ... 13:09:38, ... 13:09:39 and so on. I added an example in attachment View attachment timestamps.txt
The data itself from the CAN messages is fine and chronological, but the timestamps not. So I start to think that maybe it has nothing to do with the millisecond functions, but either with the buffering in my code or with the built-in seconds() function

 

[luni]

Couldn't you simply print the timestamp to the sermon before copying it to the message? This would show you if the problems come from your generation algorithm or some later stage.

 

[klaasdc]

Couldn't you simply print the timestamp to the sermon before copying it to the message? This would show you if the problems come from your generation algorithm or some later stage.

Late reply, the car in which this was put was not nearby so I had to wait for it to return to base to continue

Still breaking my head on this. I implemented your chrono-based method (which looks super clean btw), but I get timestamps that kind of "freeze". I'll try to illustrate this.
In the function that is called from the CAN interrupt (sd_log(), printed below), I print the actual micros() and the timestamp, and write the message to a circular buffer at the head.
In the main thread, the buffer head and tail are compared and messages are printed to the log file using the provided timestamp in the buffer.

typedef struct LogEntry{
    int year;
    int month;
    int day;
    int hour;
    int minute;
    int second;
    uint32_t millisecond;
    CAN_message_t msg;
} LogEntry;

LogEntry lbuf[CANBUFSIZE];
volatile unsigned int lbuf_head = 0;
volatile unsigned int lbuf_tail = 0;

void sd_log(const CAN_message_t* msg){
    lbuf_head++;
    if (lbuf_head >= CANBUFSIZE){
        lbuf_head = 0;
    }

    timePoint now = system_clock::now();
    time_t rawTime = system_clock::to_time_t(now); 
    tm t           = *gmtime(&rawTime);
    int year       = t.tm_year + 1900;
    int month      = t.tm_mon + 1;
    int day        = t.tm_mday;
    int hour       = t.tm_hour;
    int minute     = t.tm_min;
    int second     = t.tm_sec;
    
    int ms = duration_cast<milliseconds>(now.time_since_epoch()).count() % 1'000;

    Serial.print(micros());
    Serial.print(" ");
    Serial.printf("%d-%02d-%02d %02d:%02d:%02d.%03u", year, month, day, hour, minute, second, ms);
    Serial.print(" ");
    Serial.print(lbuf_tail);
    Serial.print("/");
    Serial.println(lbuf_head);

    lbuf[lbuf_head].second = second;
    lbuf[lbuf_head].minute = minute;
    lbuf[lbuf_head].hour = hour;
    lbuf[lbuf_head].day = day;
    lbuf[lbuf_head].month = month;
    lbuf[lbuf_head].year = year;
    lbuf[lbuf_head].millisecond = ms;

    memcpy((void*) &lbuf[lbuf_head].msg, msg, sizeof(CAN_message_t));
}

The printed values are, for example:

...
342984384 2021-09-15 21:27:06.317 2426/2427
342987384 2021-09-15 21:27:06.317 2427/2428
342988384 2021-09-15 21:27:06.317 2428/2429
342989384 2021-09-15 21:27:06.317 2429/2430
342990384 2021-09-15 21:27:06.317 2429/2431
342991384 2021-09-15 21:27:06.317 2431/2432
342992384 2021-09-15 21:27:06.317 2432/2433
342993384 2021-09-15 21:27:06.317 2433/2434
342994384 2021-09-15 21:27:06.317 2434/2435
342995384 2021-09-15 21:27:06.317 2435/2436
342997384 2021-09-15 21:27:06.317 2436/2437
342998384 2021-09-15 21:27:06.317 2437/2438
342999384 2021-09-15 21:27:06.317 2438/2439
343000384 2021-09-15 21:27:06.317 2439/2440
343001384 2021-09-15 21:27:06.317 2440/2441
...

I suspected some kind of stupid pointer problem in my buffer logic, so that old timestamps are "kept", but the printed time can stay the same for multiple minutes.
When I stop transmitting CAN messages and start again after 30s-1min, then the printed time is a new value.

Is there some pointer magic inside the chrono timepoint that could cause the reported time not to update?
Maybe my way of using the buffer is leading to memory problems? Any suggestions welcome!

 

[luni]

Is there some pointer magic inside the chrono timepoint that could cause the reported time not to update?

There is a lot of template magic but AFAIK no pointer magic. TimePoint::now simply returns a 'duration value' which consists of the value of the RTC counter (number of crystal periods) multiplied by 1E9/32768 plus some ratio info which informs the consumer that those count values need to be interpreted as nanoseconds.

I can have a closer look tomorrow evening. Here a quick trial: I commented the part which writies to your message buffer and called sdLog from an intervaltimer every 1ms.

#include "Arduino.h"
#include <chrono>

using namespace std::chrono;

// get the number of oscillator periods since 1970-01-01
uint64_t get_RTC_periods()
{
    uint32_t hi1 = SNVS_HPRTCMR, lo1 = SNVS_HPRTCLR;
    while (true)
    {
        uint32_t hi2 = SNVS_HPRTCMR, lo2 = SNVS_HPRTCLR;
        if (lo1 == lo2 && hi1 == hi2)
        {
            return (uint64_t)hi2 << 32 | lo2;
        }
        hi1 = hi2;
        lo1 = lo2;
    }
}

// this is all you need to do to prepare the built in system_clock for usage with the RTC (resolution ~30.5µs)
system_clock::time_point system_clock::now()
{
    uint64_t ns = get_RTC_periods() * (1E9 / 32768); // system clock uses ns as time base (would be better to define dedicated clock with osc periods as base)
    return system_clock::time_point(system_clock::duration(ns));
}

using timePoint = system_clock::time_point; // alias to save typing...

//---------------------------------------------------------

constexpr size_t CANBUFSIZE = 100;

typedef struct LogEntry{
    int year;
    int month;
    int day;
    int hour;
    int minute;
    int second;
    uint32_t millisecond;
    //CAN_message_t msg;
} LogEntry;

LogEntry lbuf[CANBUFSIZE];
volatile unsigned int lbuf_head = 0;
volatile unsigned int lbuf_tail = 0;

void sd_log(/*const CAN_message_t* msg*/){
    lbuf_head++;
    if (lbuf_head >= CANBUFSIZE){
        lbuf_head = 0;
    }

    timePoint now = system_clock::now();
    time_t rawTime = system_clock::to_time_t(now);
    tm t           = *gmtime(&rawTime);
    int year       = t.tm_year + 1900;
    int month      = t.tm_mon + 1;
    int day        = t.tm_mday;
    int hour       = t.tm_hour;
    int minute     = t.tm_min;
    int second     = t.tm_sec;

    int ms = duration_cast<milliseconds>(now.time_since_epoch()).count() % 1'000;

    Serial.print(micros());
    Serial.print(" ");
    Serial.printf("%d-%02d-%02d %02d:%02d:%02d.%03u", year, month, day, hour, minute, second, ms);
    Serial.print(" ");
    Serial.print(lbuf_tail);
    Serial.print("/");
    Serial.println(lbuf_head);

    lbuf[lbuf_head].second = second;
    lbuf[lbuf_head].minute = minute;
    lbuf[lbuf_head].hour = hour;
    lbuf[lbuf_head].day = day;
    lbuf[lbuf_head].month = month;
    lbuf[lbuf_head].year = year;
    lbuf[lbuf_head].millisecond = ms;

   // memcpy((void*) &lbuf[lbuf_head].msg, msg, sizeof(CAN_message_t));
}

IntervalTimer timer;

void setup()
{
  while(!Serial){}
  pinMode(LED_BUILTIN,OUTPUT);
  
  timer.begin(sd_log, 1000);
}

void loop()
{
  digitalWriteFast(LED_BUILTIN,!digitalReadFast(LED_BUILTIN));
  delay(500);
}

...
80627002 2021-09-15 22:10:03.677 0/96
280628002 2021-09-15 22:10:03.678 0/97
280629002 2021-09-15 22:10:03.679 0/98
280630002 2021-09-15 22:10:03.680 0/99
280631002 2021-09-15 22:10:03.681 0/0
280632002 2021-09-15 22:10:03.682 0/1
280633002 2021-09-15 22:10:03.683 0/2
280634002 2021-09-15 22:10:03.684 0/3
280635002 2021-09-15 22:10:03.685 0/4
280636002 2021-09-15 22:10:03.686 0/5
280637002 2021-09-15 22:10:03.687 0/6
280638002 2021-09-15 22:10:03.688 0/7
...

Stared at it for some minutes but don't observe any freezes. How long does it need to run to show the effect?

 

[klaasdc]

There is a lot of template magic but AFAIK no pointer magic. TimePoint::now simply returns a 'duration value' which consists of the value of the RTC counter (number of crystal periods) multiplied by 1E9/32768 plus some ratio info which informs the consumer that those count values need to be interpreted as nanoseconds.

I can have a closer look tomorrow evening. Here a quick trial: I commented the part which writies to your message buffer and called sdLog from an intervaltimer every 1ms.

...

Stared at it for some minutes but don't observe any freezes. How long does it need to run to show the effect?

Thanks!
The frozen time appears as soon as I start the teensy and start transmitting CAN messages from my laptop to it. I tried also reinserting the RTC battery to make sure, but no change. I let it run for a while and recorded each time the printed time would update:

23:55:59
23:58:17
00:00:34
00:02:52
00:05:09.605
00:07:27.043
00:09:44.482
00:12:01.921
00:14:19.360

So it appears that it DOES change, but only after 2m17 or 2m18s (did not check the ms in the beginning...) every time.

Also, I noticed that in Time.cpp from the core, there is a refreshCache() function. Could that have to do anything with it? It refers to 300seconds though...

static time_t cacheTime;   // the time the cache was updated
static uint32_t syncInterval = 300;  // time sync will be attempted after this many seconds

There is also a small OLED display connected to the logger, showing the time, and that is happily showing the actual time from calls to minute(), second(), hour() etc.

 

[luni]

As you also suspect, this sounds like an error somewhere else. Just to exclude some unknowns, here a version which doesn't use the chrono clocks at all. It simply reads out the RTC, calculates seconds and milliseconds from it, calls the venerable gmtime function to calculate the calendar data and stores the result in your LogEntry struct. I call it every 10ms. Runs without freezing (like the last version from above), but might be worth copying into your full sketch anyway.

I'm afraid, I can't help much more, at least the timing part of your sketch seems to work as expected. I'd have a close look at the usual suspects, i.e., array overruns, interrupt codes, possible reentrance issues....

#include "Arduino.h"
#include "time.h"

struct LogEntry
{
    int year;
    int month;
    int day;
    int hour;
    int minute;
    int second;
    uint32_t millisecond;
    //CAN_message_t msg;
};

constexpr size_t CANBUFSIZE = 100;
LogEntry lbuf[CANBUFSIZE];
volatile unsigned int lbuf_head = 0;
volatile unsigned int lbuf_tail = 0;


void getTimeStamp(LogEntry* logEntry)
{
    // read rtc (64bit, number of 32,768 kHz crystal periods)
    uint64_t periods;
    uint32_t hi1 = SNVS_HPRTCMR, lo1 = SNVS_HPRTCLR;
    while (true)
    {
        uint32_t hi2 = SNVS_HPRTCMR, lo2 = SNVS_HPRTCLR;
        if (lo1 == lo2 && hi1 == hi2)
        {
            periods = (uint64_t)hi2 << 32 | lo2;
            break;
        }
        hi1 = hi2;
        lo1 = lo2;
    }

    // calculate seconds and milliseconds
    uint32_t ms = (1000 * (periods % 32768)) / 32768;
    time_t sec  = periods / 32768;

    tm t                  = *gmtime(&sec); // calculate calendar data
    logEntry->second      = t.tm_sec;
    logEntry->minute      = t.tm_min;
    logEntry->hour        = t.tm_hour;
    logEntry->day         = t.tm_mday;
    logEntry->month       = t.tm_mon + 1;
    logEntry->year        = t.tm_year + 1900;
    logEntry->millisecond = ms;
}
//---------------------------------------------------------


void sd_log(/*const CAN_message_t* msg*/)
{
    lbuf_head++;
    if (lbuf_head >= CANBUFSIZE)
    {
        lbuf_head = 0;
    }

    LogEntry* entry = &lbuf[lbuf_head];

    getTimeStamp(entry);

    Serial.print(micros());
    Serial.print(" ");
    Serial.printf("%d-%02d-%02d %02d:%02d:%02d.%03u", entry->year, entry->month, entry->day, entry->hour, entry->minute, entry->second, entry->millisecond);
    Serial.print(" ");
    Serial.print(lbuf_tail);
    Serial.print("/");
    Serial.println(lbuf_head);

    // memcpy((void*) entry->.msg, msg, sizeof(CAN_message_t));
    //// memcpy((void*) &lbuf[lbuf_head].msg, msg, sizeof(CAN_message_t));
}


IntervalTimer timer;

void setup()
{
    while (!Serial) {}
    timer.begin(sd_log, 10000);
}

void loop()
{
}

 

[klaasdc]

As you also suspect, this sounds like an error somewhere else. Just to exclude some unknowns, here a version which doesn't use the chrono clocks at all. It simply reads out the RTC, calculates seconds and milliseconds from it, calls the venerable gmtime function to calculate the calendar data and stores the result in your LogEntry struct. I call it every 10ms. Runs without freezing (like the last version from above), but might be worth copying into your full sketch anyway.

I'm afraid, I can't help much more, at least the timing part of your sketch seems to work as expected. I'd have a close look at the usual suspects, i.e., array overruns, interrupt codes, possible reentrance issues....

#include "Arduino.h"
#include "time.h"

struct LogEntry
{
    int year;
    int month;
    int day;
    int hour;
    int minute;
    int second;
    uint32_t millisecond;
    //CAN_message_t msg;
};

constexpr size_t CANBUFSIZE = 100;
LogEntry lbuf[CANBUFSIZE];
volatile unsigned int lbuf_head = 0;
volatile unsigned int lbuf_tail = 0;


void getTimeStamp(LogEntry* logEntry)
{
    // read rtc (64bit, number of 32,768 kHz crystal periods)
    uint64_t periods;
    uint32_t hi1 = SNVS_HPRTCMR, lo1 = SNVS_HPRTCLR;
    while (true)
    {
        uint32_t hi2 = SNVS_HPRTCMR, lo2 = SNVS_HPRTCLR;
        if (lo1 == lo2 && hi1 == hi2)
        {
            periods = (uint64_t)hi2 << 32 | lo2;
            break;
        }
        hi1 = hi2;
        lo1 = lo2;
    }

    // calculate seconds and milliseconds
    uint32_t ms = (1000 * (periods % 32768)) / 32768;
    time_t sec  = periods / 32768;

    tm t                  = *gmtime(&sec); // calculate calendar data
    logEntry->second      = t.tm_sec;
    logEntry->minute      = t.tm_min;
    logEntry->hour        = t.tm_hour;
    logEntry->day         = t.tm_mday;
    logEntry->month       = t.tm_mon + 1;
    logEntry->year        = t.tm_year + 1900;
    logEntry->millisecond = ms;
}
//---------------------------------------------------------


void sd_log(/*const CAN_message_t* msg*/)
{
    lbuf_head++;
    if (lbuf_head >= CANBUFSIZE)
    {
        lbuf_head = 0;
    }

    LogEntry* entry = &lbuf[lbuf_head];

    getTimeStamp(entry);

    Serial.print(micros());
    Serial.print(" ");
    Serial.printf("%d-%02d-%02d %02d:%02d:%02d.%03u", entry->year, entry->month, entry->day, entry->hour, entry->minute, entry->second, entry->millisecond);
    Serial.print(" ");
    Serial.print(lbuf_tail);
    Serial.print("/");
    Serial.println(lbuf_head);

    // memcpy((void*) entry->.msg, msg, sizeof(CAN_message_t));
    //// memcpy((void*) &lbuf[lbuf_head].msg, msg, sizeof(CAN_message_t));
}


IntervalTimer timer;

void setup()
{
    while (!Serial) {}
    timer.begin(sd_log, 10000);
}

void loop()
{
}

Thank you luni, I appreciate it very much!
I have now started again from your code, and slowly added feature by feature and so far it is still working as it should. I noticed also that I was not servicing the FlexCAN_T4 events() fast enough and dropping CAN messages. I now put an intervaltimer at 0.1ms for that. Still have to add the LEDs and OLED functions...

But I'm intrigued by some of your changes to the coding style. I have a few small questions, if you don't mind? Just trying to improve my understanding
You put

memcpy((void*) entry->msg, msg, sizeof(CAN_message_t));

If I do that, I get an invalid cast from CAN_message_t* to void*. But, why did you change to the pointer style over the reference operator I used?

Also, why use constexpr and size_t, over a simple define, for the buffer array size?

Thanks again for your time!

 

[luni]

But, why did you change to the pointer style over the reference operator I used

. No deep deep thought behind that. I'm just used to pass structs by address into functions. Passing by reference would work the same of course. (and, probably the real reason: on a german keypboard 'entry->' is much easier to type than 'lbuf[lbuf_head]'

If I do that, I get an invalid cast from CAN_message_t* to void*

. Sorry just wrote this in the comment without testing....

Also, why use constexpr and size_t, over a simple define, for the buffer array size?

Using #defines (or enums) for constants was necessary in C since C has no other way to generate compile time constants (const is more like readonly). The c++ constexpr guarantees a true and typed compile-time constant. So, there is no need to use the preprocessor to generate a constant anymore. For some reason which I never understood you still see them a lot in embedded programming. I made a habit of just avoiding them. If you are interested, there are lots of discussions about constexpr vs defines in the usual places (e.g. https://stackoverflow.com/questions/42388077/constexpr-vs-macros).

Another nice bonus: compare the information you get from modern IDEs when you e.g. hover over a #define vs a constepxr. Hovering over PERIOD shows:



Doing the same with a constepxr shows:

 

[jmlima44]

Hi I am trying to implement a data logging system with a timestamp. I am already saving successfuly the data in our SD card and I am using a teensy 4.1 microcontroller. But, when I tried to implement the timestamp, nothing worked. Can you help me? This is my code:
In the code, LogEntry is a struct that is defined in the logging.h file.

#include "logging.h"

extern FlexCAN_T4<CAN1, RX_SIZE_256, TX_SIZE_16> can1;
File myFile;

LogEntry *entry;

void getTimeStamp(LogEntry* logEntry)
{
    // read rtc (64bit, number of 32,768 kHz crystal periods)
    uint64_t periods;
    uint32_t hi1 = SNVS_HPRTCMR, lo1 = SNVS_HPRTCLR;
    while (true)
    {
        uint32_t hi2 = SNVS_HPRTCMR, lo2 = SNVS_HPRTCLR;
        if (lo1 == lo2 && hi1 == hi2)
        {
            periods = (uint64_t)hi2 << 32 | lo2;
            break;
        }
        hi1 = hi2;
        lo1 = lo2;
    }

    // calculate seconds and milliseconds
    uint32_t ms = (1000 * (periods % 32768)) / 32768;
    time_t sec  = periods / 32768;

    tm t                  = *gmtime(&sec); // calculate calendar data
    logEntry->second      = t.tm_sec;
    logEntry->minute      = t.tm_min;
    logEntry->hour        = t.tm_hour;
    logEntry->day         = t.tm_mday;
    logEntry->month       = t.tm_mon + 1;
    logEntry->year        = t.tm_year + 1900;
    logEntry->millisecond = ms;
}


void Logging::setup_log() {

    //SPI.setCS(PIN_SD_CS);    
    //SPI.setMISO(PIN_SPI_MISO);
    //SPI.setMOSI(PIN_SPI_MOSI);
    //SPI.setSCK(PIN_SPI_SCK);
    // Setup pinout
    pinMode(PIN_SPI_MOSI, OUTPUT);
    pinMode(PIN_SPI_MISO, INPUT);
    pinMode(PIN_SPI_SCK, OUTPUT);
    // Disable SPI devices
    pinMode(PIN_SD_CS, OUTPUT);
    digitalWrite(PIN_SD_CS, HIGH);


    // Setup serial
    Serial.begin(9600);

    // Setup SD card
    if (!SD.begin(BUILTIN_SDCARD)) {
        Serial.println("initialization failed!");
        return;
    }

}
void Logging::write_to_file(int current, int voltage, int mintmp, int maxtmp, int avgtmp, int apps1, int apps2, int brake) {
    
        //Serial.print("Starting to write...");

        myFile = SD.open("Test01.txt", FILE_WRITE);

        getTimeStamp(entry);

        myFile.printf("%d-%02d-%02d %02d:%02d:%02d.%03u \n", entry->year, entry->month, entry->day, entry->hour, entry->minute, entry->second, entry->millisecond);


        myFile.printf("Current - %d \n",current);

        myFile.printf("Voltage - %d \n",voltage);

        myFile.printf("MinTmp - %d \n",mintmp);

        myFile.printf("MaxTmp - %d \n",maxtmp);

        myFile.printf("AvgTmp - %d \n",avgtmp);

        myFile.printf("Apps1 - %d \n",apps1);
        
        myFile.printf("Apps2 - %d \n",apps2);

        myFile.printf("Brake - %d \n",brake);


        myFile.close();
}

 

[sicco]

This worked for me, writing a 42 bits atomic millisecond resolution timestamp to a string as 32 bit seconds part plus 10 bits millisecond:

void GetTimeDateNowAsString3 (char *st)
{
    // read rtc (64bit, number of 32,768 kHz crystal periods)
    uint64_t periods;
    uint32_t hi1 = SNVS_HPRTCMR, lo1 = SNVS_HPRTCLR;
    while (true)
    {
        uint32_t hi2 = SNVS_HPRTCMR, lo2 = SNVS_HPRTCLR;
        if (lo1 == lo2 && hi1 == hi2)
        {
            periods = (uint64_t)hi2 << 32 | lo2;
            break;
        }
        hi1 = hi2;
        lo1 = lo2;
    }

    // calculate seconds and milliseconds
    uint32_t ms = (1000 * (periods % 32768)) / 32768;
    time_t timenow = periods / 32768;

    sprintf (st, "%ld.%03d", (long)timenow, (int)ms);
}

 

[jmlima44]

This worked for me, writing a 42 bits atomic millisecond resolution timestamp to a string as 32 bit seconds part plus 10 bits millisecond:

void GetTimeDateNowAsString3 (char *st)
{
    // read rtc (64bit, number of 32,768 kHz crystal periods)
    uint64_t periods;
    uint32_t hi1 = SNVS_HPRTCMR, lo1 = SNVS_HPRTCLR;
    while (true)
    {
        uint32_t hi2 = SNVS_HPRTCMR, lo2 = SNVS_HPRTCLR;
        if (lo1 == lo2 && hi1 == hi2)
        {
            periods = (uint64_t)hi2 << 32 | lo2;
            break;
        }
        hi1 = hi2;
        lo1 = lo2;
    }

    // calculate seconds and milliseconds
    uint32_t ms = (1000 * (periods % 32768)) / 32768;
    time_t timenow = periods / 32768;

    sprintf (st, "%ld.%03d", (long)timenow, (int)ms);
}

But This was too implement only the miliseconds, my goal is to print into my file (in the SD card) the full timestamp: year,day,hour,minutes,seconds and miliseconds. I can't understand why my work doesn't work when I try to implement my getTimsestamp() function. If I try the data logging without the timestamp everythign works, but it is when I call the function that nothing gets written. My observation is that there is an error in my auxiliary function (the one it calculates the time) but I can't figure out why. Is it necessary to connect something specific to the teensy 4.1 in order to make the RTC work? He is connected to my formula student's team car and works fine, but is there any specific pin to make the RTC work?

 

[sicco]

But This was too implement only the miliseconds, my goal is to print into my file (in the SD card) the full timestamp: year,day,hour,minutes,seconds and miliseconds. I can't understand why my work doesn't work when I try to implement my getTimsestamp() function. If I try the data logging without the timestamp everythign works, but it is when I call the function that nothing gets written. My observation is that there is an error in my auxiliary function (the one it calculates the time) but I can't figure out why. Is it necessary to connect something specific to the teensy 4.1 in order to make the RTC work? He is connected to my formula student's team car and works fine, but is there any specific pin to make the RTC work?

Difficult to say without seeing the full sketch source code.

You have this as pointer declaration

LogEntry *entry;

And then use this later on

    logEntry->second      = t.tm_sec;
    logEntry->minute      = t.tm_min;
    logEntry->hour        = t.tm_hour;
    logEntry->day         = t.tm_mday;
    logEntry->month       = t.tm_mon + 1;
    logEntry->year        = t.tm_year + 1900;
    logEntry->millisecond = ms;

But where is the memory space declared that you're writing that into? Should that not also have a global parameter declaration like

LogEntry globalLogEntry;
LogEntry *entry = &globalLogEntry;

 

[jensa]

This is an old thread, but I found it looking for an RTC with resolution smaller than a second, so I thought I'd update with new info. My usecase is to make a device wake up from deep sleep every 250ms and do exactly 4 samples each second. After sleeping to conserve power, the device will not know anything about how long it's been sleeping. Also - the code will not take exactly the same time to execute every time, so I have to compensate the sleep time every time the device wakes up.

OP asked if there was anything that had higher resolution than a second and I've found that the DS1390 (SPI) and DS1388 (I2C) actually has hundredths of a second (Centisecond) resolution, so I'll implement one of these in my next design. They can apparently also trickle charge the RTC battery and has some builtin EEPROM storage, so they might be a nice upgrade from classic devices such as DS3231 and DS1307. Comparable power consumption as well.