Over the air updates

[jonr]

Sometimes an application requires teensy firmware updates without access to a PC and the USB port. So here is a serial port/hex program loader that is ready for alpha testing/feedback/fixes. Basic algorithm: load new program to upper half of flash memory then copy to lower flash memory and then reboot. Easily modified to work with any serial data stream (bluetooth, wifi, etc). Has integrity checks - it won't brick the MCU if a update isn't fully loaded.

Note, when dealing with flash, it is always possible to permanently ruin your teensy - don't run this unless you are willing to risk that; with your only recourse being to buy a new teensy.

See the flasher.* code here: https://github.com/Photosynq/Photosy...ter/multispeq1

Note: version 1.2 tested on teensy 3.0, 3.1 and 3.2. Don't try it on anything else.

tags: bootloader, OTA, flash

[pictographer]

My two cents:

1. Cool! Thanks!
2. Would be good to add validity checks. Somehow the Teensy loader can detect whether or not a hex file was compiled for the connected Teensy. Here is some code that can detect what processor it is running on and what the flash size is (Warning: this code needs an expert review): http://pictographer.com/didah/teensy...pp_source.html
3. Putting it up on Github would make collaboration easier.
4. Dividing the code into an example and a library would be nice.
5. I saw there was some code to avoid stepping on magic values in the flash. Does this include the mac address or just the bits vital to avoid bricking the device?

Again, thanks!

[stevech]

because the teensy downloader code erases all of flash (retaining a few vital areas), is the concept for "flasher" that it is always used as the app code updater? And without a custom ld (linker script) how do you put the flasher code in a reserved address space/sector?

Nice work.

[jonr]

Good comments, thanks.

The idea is that the flasher code is part of your application - no reserved flash area or ld changes. Initially you load your application with the teensy MINI54 loader, but updates can be sent over the serial port.

[jonr]

I suppose a string could be embedded in the new program and flasher could check that it is there somewhere. I'm not aware of any other way to look at a hex file and determine if it will run on any particular MCU.

[Frank B]

The teensy-loader looks into the elf file (if present) to verify the mcu, if i remember correctly.
You could check the "magic bytes" in block 0. At least you can check if it is a file for teensy 3.x this way.
Would it make sense to add hex-file reading to my sd-card bootloader ? Is here anyone who uses it ? :-)

[jonr]

For those not familiar with it, Frank's nice program is similar, but loads a new program from a SD card. It was inspirational in that when I had a bug, I knew a fix was possible (although unfortunately my bug pertained only to the teensy 3.0).

[jonr]

OK, I added code to check that the firmware being uploaded includes the code that will allow future uploads.

[Koromix]

In my own code I check that the HEX file is the correct one by checking for "machine code" signatures, which correspond to model-specific and so far conservated code segments in Paul's core code.

It's very naive, but it works well enough for me and so far, and has yet to cause any problem (survived several GCC and Teensyduino versions unchanged). The good part is, you don't need the ELF file to do that. Here is the relevant part of my code (won't compile as is).

struct firmware_signature {
    const tyb_board_model *model;
    uint8_t magic[8];
};

static const struct firmware_signature signatures[] = {
    {&_tyb_teensy_pp10_model, {0x0C, 0x94, 0x00, 0x7E, 0xFF, 0xCF, 0xF8, 0x94}},
    {&_tyb_teensy_20_model,   {0x0C, 0x94, 0x00, 0x3F, 0xFF, 0xCF, 0xF8, 0x94}},
    {&_tyb_teensy_pp20_model, {0x0C, 0x94, 0x00, 0xFE, 0xFF, 0xCF, 0xF8, 0x94}},
    {&_tyb_teensy_30_model,   {0x38, 0x80, 0x04, 0x40, 0x82, 0x3F, 0x04, 0x00}},
    {&_tyb_teensy_31_model,   {0x30, 0x80, 0x04, 0x40, 0x82, 0x3F, 0x04, 0x00}},
    {&_tyb_teensy_lc_model,   {0x34, 0x80, 0x04, 0x40, 0x82, 0x3F, 0x00, 0x00}},
    {0}
};

// Give it a firmware, get the model it was compiled for (or NULL)
const tyb_board_model *tyb_board_test_firmware(const tyb_firmware *f)
{
    assert(f);

    if (f->size < sizeof(signatures[0].magic))
        return NULL;

    /* Naive search with each board's signature, not pretty but unless
       thousands of models appear this is good enough. */
    for (size_t i = 0; i < f->size - sizeof(signatures[0].magic); i++) {
        for (const struct firmware_signature *sig = signatures; sig->model; sig++) {
            if (memcmp(f->image + i, sig->magic, sizeof(signatures[0].magic)) == 0)
                return sig->model;
        }
    }

    return NULL;
}

Of course, I can't guarantee that it's future-proof.

[Frank B]

I'm seeking beta testers who want to test hex-file flashing from SD card.
Experienced teensy-users only!

[stevech]

I'm seeking beta testers who want to test hex-file flashing from SD card.
Experienced teensy-users only!

Thread title "over the air updates"... with the from-SD card update, the re-flashing code cares not where how to code was put onto the SD card, right?
Over the air wireless (WiFi, sub-GHz),
Over the serial,
Over the Ethernet,
Over the xxx,
etc.
Right?

Since the re-flasher takes over the MCU (yes?) to block/own interrupts, relocating itself to RAM? - Does it use the same FATFS that the calling app uses? Or is there some way to know the cluster list sectors that comprise the desired file and thus not use the FATFS per se?

[Frank B]

Yes, it does not matter how the file was written to the card. It is in the standard filesystem. Maximum flashable size is half of the Teensy 3.1 flash.

[stevech]

.... Maximum flashable [application program's] size is half of the Teensy 3.1 flash.

why would this limitation apply when the new code is in SD, rather than temporarily stored in upper flash space? (excluding the top sector or two)?

lemme try to answer my own question, above....
It's so that a small piece of code running in RAM can read new code stored in top half of flash and write to bottom half - without needing SD drivers or FATFS or interrupts?
(no power failures allowed!)

[Frank B]

Thats because this loader is part of the application.
That's the way it works. It loads a the complete hex file into the free flash, then a small part which runs in ram (and with disabled interrupts) overwrites the current application with this copy.

Indeed, there are other ways to do this - but this is a very simple approach and very compatible without changing anything inside arduino or linker files.
It it possible to change it ( i have an idea in mind, but this is not the time to talk about it) in a later version.

But, for now, if you need more than 128 KB for your application, you should use another solution (i.e. utasker)

[jonr]

I suspect that few teensy programs exceed 128K. But one could load an intermediate program into ram and then run that to leave all of the flash available for a new program. Should just be some minor changes to the teensy linker files to create an all ram program.

[stevech]

Thats because this loader is part of the application.
That's the way it works. It loads a the complete hex file into the free flash, then a small part which runs in ram (and with disabled interrupts) overwrites the current application with this copy.

Indeed, there are other ways to do this - but this is a very simple approach and very compatible without changing anything inside arduino or linker files.
It it possible to change it ( i have an idea in mind, but this is not the time to talk about it) in a later version.

But, for now, if you need more than 128 KB for your application, you should use another solution (i.e. utasker)

Simple is good, in many cases!
Thanks for the contribution.

[schmo]

hy can you upload a full working simple example with a working programm like LED_BLINK.ino or something like that?

[jonr]

That's really what the uploaded file is. Let me know if it works for you.

[schmo]

so i tried it again.

i compiled the led blinking teensy sample to a hex file with the standard arduino ide.

now i connect to the teensy with putty

start the uploading,
and before i can send :flash 770

i get the message "new firmware not compatible"

[schmo]

ok after changing the line
if (check_compatible(min_address + FLASH_SIZE / 2 , max_address + FLASH_SIZE / 2)) to
if (check_compatible(min_address , max_address + FLASH_SIZE / 2))

it finally works like charm :-)

i steped into to hex file and saw that the teensy Flash id is stored at line 50 so the minadress range was to high.

Now i am able to rewrite the whole thing to work with a can-bus

[jonr]

Normal behavior is to only allow an update to new firmware that will also allow further updates. But yes, you can disable this feature.

[russdx]

I am trying to get this to work with no luck. I can send the file and receive the
"done, 770 hex lines, address range 0:2ff8, waiting for :flash 770"
message, But it never seems to actually run my blinky sketch. Just dies at this point? Never seems to install the new hex.

Ah, Got it working you have to type :flash 770 twice?

Ah, Got it working with just typing it once, must of added extrx /n or something to the hex code.

Nice little program

[onehorse]

I wonder if this will work with the ESP8266?

[dheerajdake]

Hello all,
I am using ESP8266 UART stream to update attached Teensy ++2.0 module. So far, i did the following,

1. I converted the hex file to bin file and uploaded it on ESP8266 flash
2. I have set Teensy into bootloader mode
3. ESP8266 and Teensy are connected through UART

After setting Teensy into bootloader mode, if i send the bin data stream through UART, will the firmware update work? The current bootloader is the halfkay bootloader. Is it designed only to work with the USB?

[someteen]

Does anyone managed to upload a sketch over the air using a Teensy 3.2 + ESP8266 combo (or a RF24 one for that matter)??