Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 6 of 6

Thread: Detect when RTC has been "automatically" set (by loader or other)?

  1. #1
    Junior Member
    Join Date
    Jun 2019
    Posts
    3

    Detect when RTC has been "automatically" set (by loader or other)?

    Hello,

    I'm absolutely loving the Teensy line of products -- they are solving so many immediate needs for our projects that used to require a number of special-purpose boards.

    We're wanting to use the Real Time Clock (RTC) in several of our applications, all on Teensy 3.5 or 3.6 boards with a CR2032 backing up the RTC. However, the Teensy loader / startup coade automatically setting the time can get in our way. I know that for most applications, having the system "automatically" initialize the RTC when code is loaded is incredibly user-friendly, but for our purposes it gets in the way. For one, the timestamp coming from the loader appears to be technically incorrect -- storing a "time_t" value that is offset by the compiler's local timezone, instead of using direct UTC values that time_t is supposed to have (per the C standard, and what we want to log/record).

    For our needs, it would be far better for an unset/automatically set RTC to give us clearly erroneous values (e.g., pre-2000 dates) that flag that it is not the true time than to give us values that are offset from the true time but not detectably/obviously so.

    In short, if the RTC value hasn't been explicitly set by our software but has a value coming from the loader, we'd like to have a way to know that.

    Here's what I've surmised from Paul's *excellent* post in https://forum.pjrc.com/threads/47988...l=1#post160639 and other forum posts, and where our specific challenge with the current system is...

    I understand that once a battery is hooked up to vBat and the time is set (either by sketch or by the loader), then the loader/startup code won't subsequently modify the RTC value as long as power exists to vBat. Excellent. Our problem arises that technicians in the field may end up disconnecting vBat, either accidentally or purposefully. Then when things are restarted, the RTC gets a value from a sketch and it's not obviously wrong.

    On the first reboot after losing vBat, my understanding is that the startup code sets the RTC to the timestamp of the currently loaded sketch, and puts a special value into vBat memory so we know that it's "wrong". Okay, we can use that. Our sketches could look at the vBat memory location and know that the clock is "off" and flag things accordingly.

    However, if while in this "stale" state the technician also loads a new sketch onto the Teensy (e.g. for a software update), then the loader causes the RTC to be initialized with a value coming from the compiler and clears the vBat memory flag. Now we're in trouble, because it appears to the sketch as though the RTC clock is set (the flag has been cleared), but it unfortunately has a value that isn't a true time_t value (it's offset by the timezone of the loader). So our logs/programs start using/logging timestamps that aren't actually UTC.

    For a variety of reasons we don't want timezone-offset values in the RTC clock; we want to use UTC time_t values only. For one, keeping the offset in the 32-bit "time_t" value is very non-standard -- it won't work with most Unix tools and C-libraries; for another, it doesn't play well with DST changes, which are also important to us.

    If there's a way to prevent the loader from ever setting the RTC clock, that'd likely work for us. Or if there's a non-volatile flag or something that can be queried in a sketch to determine if the RTC contains an "automatic" value, that'd work great also.

    If there's not a way in the existing configuration to determine the RTC state, then I'm thinking we may have to use some of the unused vBat memory to store our own flag value to say "yes, we've explicitly set the clock", similar to the way the startup code is currently using 0x4003E01C. But I'd prefer to take that approach as a last resort, since it's poking around in storage not specifically allocated to us for that purpose.

    Another approach might be to have our sketches look at the 0x4003E01C location and if it's the special startup code flag value, then explicitly set the RTC clock to a "clearly impossible" (pre-2000) date so subsequent loads don't change it. But that assumes that our sketches are the only ones that get loaded onto the Teensy; if another sketch (e.g. Blink) is loaded for testing purposes, then there's a chance the loader will end up setting the clock and our values are incorrect again.

    Are there other ways of detecting the "automatic initialization" of the RTC that I haven't found yet?

    Thanks again for such a wonderful product.

    Pm

  2. #2
    Senior Member
    Join Date
    Nov 2015
    Location
    Cold hollow VT
    Posts
    118
    Why not just have the time on the PC/MAC that programs the Teensy set to UTC, instead of local time? I understand it is not what you asked, but I see it as achieving the same end goal. Otherwise add a serial routine to set the RTC manually.

  3. #3
    Junior Member
    Join Date
    Jun 2019
    Posts
    3
    Quote Originally Posted by grease_lighting View Post
    Why not just have the time on the PC/MAC that programs the Teensy set to UTC, instead of local time? I understand it is not what you asked, but I see it as achieving the same end goal. Otherwise add a serial routine to set the RTC manually.
    Thanks for the reply. I don't generally or necessarily control the computers the technicians could be using to program the Teensys. it's not like it's "the PC/MAC" (singular) that may be programming them. When a technician is remote in the field, I may need to email them some code that they load onto the Teensy using whatever laptop/device they have with them. If I also say "oh, you have to reset your machine to be in the UTC timezone"... that's just another step or place for things to go wrong.

    It also turns out that (on my machine at least, running Kubuntu), you have to be sure the timezone is set before you start the IDE/Teensy loader application. So it then requires a reboot and... why put all of this complication on the user side? I think it's better for systems (the Teensy running our project in this case) to adapt to the users/technicians than to make everyone adapt to the system.

    It shouldn't be all that hard for the Teensy environment to have some way of detecting (or suppressing) automatic setting of the RTC, so that's the more robust path.

    Pm

  4. #4
    Senior Member
    Join Date
    Nov 2015
    Location
    Cold hollow VT
    Posts
    118
    This is what sets the RTC at program time.

    Code:
      //  --------------------------------  deal with RTC stuff
      // set the Time library to use Teensy 3.0's RTC to keep time
      setSyncProvider(getTeensy3Time);
      delay(100);
      if (timeStatus()!= timeSet) {
        Serial.println("Unable to sync with the RTC");
      } else {
        Serial.println("RTC has set the system time");
      }
      Teensy3Clock.set(now());        //  comment out to not set RTC at program time
    Simply don't execute the last line and the Teensy will not get time set from the code source.

    I use this to set time and date from the serial event handler.

    Code:
      void setTD() {
    
    //        Serial.print("ARGS: ");
            strncpy(werk, args.c_str(), sizeof(werk));      //  convert STRING to CHAR array
            Serial.println(werk);
    
            hours = hour();         //  define some defaults to eliminate need for all variables
    //        Serial.println(hours);
            minutes = minute();
            seconds = second();
            days = day();
            months = month();
            years = year();
            p1 = 0;                 //  need to parse input string  p3 = length p1 as pointer p4 as item pointer
            p2 = 0;
            p3 = strlen(werk);
            p4 = 1;
            tmp[0]=0;
            strcpy(units,"<NONE>");
            if (werk != units) {                   //  abort if null time-date info
            do  {                                   //  <sp> <:> <-> </> point to next item
              if ((werk[p1]==' ') || (werk[p1]==':') || (werk[p1]=='-') || (werk[p1]=='/') || (p1>=p3) )  {                   
    //            Serial.println("Found delimiter");
                if (p4==1)  {hours=atoi(tmp);}                                        //  save value in proper item variable
                if (p4==2)  {minutes=atoi(tmp);}
                if (p4==3)  {seconds=atoi(tmp);}
                if (p4==4)  {days=atoi(tmp);}
                if (p4==5)  {months=atoi(tmp);}
                if (p4==6)  {years=atoi(tmp);}
    //            Serial.print("P1: ");
    //            Serial.println(p1);
    //            Serial.println(tmp);
                tmp[0]=0;                                                     //  clear for next item
                p4++;   //  increment item pointer
                p2=0;
              }   else  {  tmp[p2] = werk[p1]; p2++; tmp[p2]='\0';}           //  build value       
              p1++;
              if (p1>p3) {p4=-1;}                                                    //  exit condition
            } while (p4 > 0);
            }   //  end  <NONE>
    //      Serial.println("DONE ! ");
    //      Serial.println(hours);
    //      Serial.println(minutes);
    //      Serial.println(seconds);
    //      Serial.println(days);
    //      Serial.println(months);
    //      Serial.println(years);
    
          setTime(hours,minutes,seconds,days,months,years); // Another way to set the time
    
      }   //  end setTD

  5. #5
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,598
    Quote Originally Posted by pmichaud View Post
    If there's a way to prevent the loader from ever setting the RTC clock, that'd likely work for us.
    As of Teensyduino 1.46, the Teensy Loader does not actually set the RTC. However, we are working on having it do this for Teensy 4.0, and perhaps in some future version we may bring that feature to Teensy 3.x boards too. But at least right now, Teensy Loader is not directly setting the RTC.

    Instead, what appears to be Teensy Loader doing this is actually a small chunk of code in the startup code in mk20dx128.c starting at line 1125. You're already seeing the side effect of its storage of the flag in the RTC's memory. This code, not Teensy Loader, is what's actually responsible for the automatic RTC time set. Well, also Arduino's ability to embed the build time time into a linker symbol named "__rtc_localtime". Teensy Loader doesn't (currently) touch the RTC at all. It simply loads a binary image, which embeds its built time and has carefully crafted startup code to set the RTC using that build time.

    One minor flaw in the scheme is the lag between the linker creating the binary image and the moment this code actually runs. Between those moments are USB enumeration for the HID-based bootloader, erase of prior code, uploading the new program, and other various delays. For a large program and on a slow PC running Windows 7 (which is much slower at USB enumeration), those times can all add up to a few seconds of error. On Teensy 4.0, we're putting the capability to have Teensy Loader actually set the RTC, as you've imagined it must be working, with the main goal of reducing the lag so the RTC ends up with exactly the same time as your PC.


    If there's not a way in the existing configuration to determine the RTC state, then I'm thinking we may have to use some of the unused vBat memory to store our own flag value to say "yes, we've explicitly set the clock", similar to the way the startup code is currently using 0x4003E01C. But I'd prefer to take that approach as a last resort, since it's poking around in storage not specifically allocated to us for that purpose.
    This is probably the best approach, based on what I can see of your explanation of your application.

    I can't see any reason why you should shy away from using those other 12 bytes of RTC RAM for your own purpose. Just think of them like all the rest of the RAM on this chip. The startup code's use of those 4 bytes is a little unusual, but keep in mind it's not Teensy Loader using the memory. It's the startup code.


    For one, the timestamp coming from the loader appears to be technically incorrect -- storing a "time_t" value that is offset by the compiler's local timezone, instead of using direct UTC values that time_t is supposed to have (per the C standard, and what we want to log/record).
    Whether to use UTC versus local time has come up before.

    Until some (likely distant) future where the Time library supports time zones, we're going to stick with local time. Ease of use for novices is the main design goal. Experts who need and understand UTC can deal with the offset. If you're wanting UTC, I know that's not the answer you want to hear. I do understand how UTC is technically the correct way. Usability is the overriding goal, even at the expense of technical rigor.

    I might also mention, the Arduino devs have talked several times about putting a time API into Arduino's core library. That would make the Time library obsolete. Of course Teensyduino will strive for best possible compatibility with Arduino's APIs. If they choose to work with UTC and implement time zones, that would also prompt Teensyduino to change to UTC. But if Arduino decides to use local time in their (likely very far future) official time API, that would forever lock in use of local time.

  6. #6
    Junior Member
    Join Date
    Jun 2019
    Posts
    3
    Quote Originally Posted by PaulStoffregen View Post
    As of Teensyduino 1.46, the Teensy Loader does not actually set the RTC. However, we are working on having it do this for Teensy 4.0, and perhaps in some future version we may bring that feature to Teensy 3.x boards too. But at least right now, Teensy Loader is not directly setting the RTC.
    Thank you so much for the speedy and nicely detailed reply. I was really intending "loader sets the RTC" as being something that happened either directly or indirectly (e.g. by the build timestamp stored in "__rtc_localtime"), but the clarification and information on future plans is very helpful.

    This is probably the best approach, based on what I can see of your explanation of your application.

    I can't see any reason why you should shy away from using those other 12 bytes of RTC RAM for your own purpose. Just think of them like all the rest of the RAM on this chip. The startup code's use of those 4 bytes is a little unusual, but keep in mind it's not Teensy Loader using the memory. It's the startup code.
    Thanks, this is the approach we'll use. As long as it's relatively "safe" for us to use that memory, this gives us all the control we'll need. Is it only 12 bytes of RTC RAM available? I thought I read there was more than that (perhaps for another model)? At any rate, this space is plenty for our needs.

    Until some (likely distant) future where the Time library supports time zones, we're going to stick with local time. Ease of use for novices is the main design goal. Experts who need and understand UTC can deal with the offset. If you're wanting UTC, I know that's not the answer you want to hear. I do understand how UTC is technically the correct way. Usability is the overriding goal, even at the expense of technical rigor.
    Yes, and I completely agree with Teensy's design choices here. A fully-functioning timezone-supporting library is a pretty hefty chunk of code, and overkill for what the vast majority of users will need. Keeping a localtime-based timestamp certainly works. My only challenge was that there didn't seem to be a way to easily maintain the C-standard UTC-based way of doing things.

    Thanks again for an awesome environment.

    Pm

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •