'Over the Air' firmware updates, changes for flashing Teensy 3.5 & 3.6

It would be really nice if the flasher code could directly accept any .hex lines it is likely to receive.
 
Agree, Jon. That's at the top of my list for the next update to Flasher. Better to integrate what I did with IntelHex directly into Flasher.
 
I working on this some more and looking into the code compiled I see this about middle of the code (about 129k total code size)

:10FFE000704700BF18EA0100054A02608A7D02716D..
:10FFF000CA7D42710A7E82714A7EC271704700BF1B..
:020000021000EC.. :1000000018EA0100064A02608A7
F0271CA7F4271C3..
does the compiler do this and is there any way to tell the compiler not to do this?
This happens twice in the 129k code block.
I can't manually fix this as it would mess all the following address's, off by 2 in this case.
Unless all the following address jumps are by reference and not absolute.
 
Don't worry about the ":02 0000 02" lines, they aren't code values. But for now, do manually combine any ":04 XXXX 00" lines.
 
I was able to get it working. Flasher3b has no problem with 130k code size. uploaded and works.
Thanks for all the help!
Still some more to do, make code with the flasher and test.
 
Flasher3c.zip is attached. This update accepts the as-built Teensy hex files and so avoids the need to pre-process to get 64-bit aligned hex records for T35/T36. The differences from Flasher3b are (a) flash and LMEM functions are broken out into a separate file (FlashT3x.cpp/h), and (b) the flash_block() function in FlashT3x.cpp has new static variables and logic to buffer bytes in T35/T36 hex records so that all flash writes are 64-bit aligned. The logic is based on the following observations of Teensy hex files:

- all hex records are 32-bit aligned
- all hex records are sequential, i.e. there are no address "gaps"
- if the last 32-bit word is on a 64-bit boundary, its value is always 0xFFFFFFFF

flash_block() confirms the first two conditions. Assuming all are true, flash_block() needs to buffer either 0 or 4 bytes. If 4 bytes from the previous record have been buffered, the new bytes are appended, the write address is decremented by 4, and as many 64-bit phrases as possible are written. If 4 bytes are left over, they are retained for the next pass.

Frank, give this a try with your large hex file and let us know how it goes. As long as the upper/lower FSEC fields match, it should be safe to do the flash update.
 

Attachments

  • Flasher3c.zip
    9.7 KB · Views: 104
Great stuff! It works with my file 8120 lines x 16 bytes = 129920 bytes code size flashed with no problem.
Code seems to work as expected, will test and report.
Joe thanks for making this work. now we have a way to do updates remotely.
 
Thanks for letting us know, Frank. That's good news. I'd like to understand why the hex files always have the same pattern of record lengths ...16,16,16,8,4,16,16,16... There must be reason those records of length 8 and 4 always occur. It's the 4-byte record that moves all subsequent records off of the 64-bit boundary. If that can be "fixed" via the the linker file or something in the code, it would be well worth the cost of 4 bytes of padding.

One correction I wanted to mention from my previous message. I don't think they occur, but address "gaps" are okay as long as they occur between hex records and on a 64-bit boundary. For example, if the first hex record defined addresses 0-15, and the second skipped ahead to 32-47, that would not be flagged as an error. An error occurs if there are 4 bytes in the static buffer, waiting to be written, and the record that follows does not define the bytes immediately following those in the buffer.
 
The hex lines below from a T3.6 hex file show the 4-byte record at AD78 that breaks the 64-bit alignment. Further below are lines from the symbol file. AD78 is .fini (language cleanup code), which is followed by arm.exidx, which has something to do with the exception table. I experimented with specifying 8-byte alignment, but it causes linker errors. I'm going to leave this alone.

:10AD60005FF800F0D109FF1F5FF800F05109FF1FE5
:08AD7000815B000035040000C6
:04AD7800F8B500BF6B
:08AD7C0068F3FF7F01000000F5
:10AD840072B60D4B41221A7003F8010C073B802266

SYMBOL TABLE:
00000000 l d .text 00000000 .text
0000ad78 l d .fini 00000000 .fini
0000ad7c l d .ARM.exidx 00000000 .ARM.exidx
 
Ya I get that kind of break up too.
The first two 0x08 and 0x04 are easy to combine. but at only 12 bytes would need to grab 4 bytes from the next line too. just need to compute the right checksum.
:04 AD78 00 F8 B5 00 BF 6B line is going to write 0xBF to AD7C and the next line
:08 AD7C 00 68 F3 FF 7F 01 00 00 00 F5 looks like it will over write AD7C with 0x00, strange I thinks.
Need to figure out how the compiler does this. It's just me 'need to know syndrome'
No biggie as your flasher3c works like a charm. We're go to good.
 
Last edited:
Hello joepasquariello! Many thanks for your work!
For a long time I have been looking for the possibility of updating the firmware in T35 from the SD card. Your Flasher looks promising, but i dont understand some moments:

1. How to upload hex file? In your scetch said // code to allow firmware update over any byte stream (serial port, radio link, etc), please tell about it in more detail, if i read file from SD
Code:
File entry = SD.open(filename.c_str(), FILE_READ);
entry.read(filebuffer, MAX_BUFFER_SIZE);
how can i flash this stream, or it is impossible?

2. In your scetch said // Max size is 1/2 of flash
Arduino compiller said me that my scetsh for T35 is 295188 bytes (56%) and my hex file is 768kB. I dont understand, what size should I use? Is this restriction temporary?
// TODO Modify to allow any combination of programs that fit in flash - ie, allow a 64K program to load a 188K program

PS. Sorry for my bad english and in advance thanks for your answers
 
Update
1. I have solved this problem, work fine
This is my code
Code:
void upgrade_firmware(void)   // main entry point
{
  Serial.printf("%s flash size = %dK in %dK sectors\n", FLASH_ID, FLASH_SIZE / 1024, FLASH_SECTOR_SIZE / 1024);

  flash_erase_upper ();   // erase upper half of flash

  if ((uint32_t)flash_word < FLASH_SIZE || (uint32_t)flash_phrase < FLASH_SIZE
      || (uint32_t)flash_erase_sector < FLASH_SIZE || (uint32_t)flash_move < FLASH_SIZE) {
    Serial.printf("routines not in ram\n");
    return;
  }

  // what is currently used?
  int32_t addr = FLASH_SIZE / 2;
  while (addr > 0 && *((uint32_t *)addr) == 0xFFFFFFFF)
    addr -= 4;
  Serial.printf("current firmware 0:%x\n", addr + 4);

  if (addr > FLASH_SIZE / 2 - RESERVE_FLASH) {
    Serial.printf("firmware is too large\n");
    return;
  }

  Serial.printf("WARNING: this can ruin your device\n");
  Serial.printf("waiting for intel hex lines\n");

  char line[200];
  int count = 0;
  int c;
  // read in hex lines

  for (;;)  {
    if (SD.exists("test1.hex")) {
      Serial.printf("file found\n");
      File entry = SD.open("test1.hex", FILE_READ);
      if (entry) {
        Serial.printf("file opend\n");
        my_line_count = 0;
        while (entry.available()) {
          c = entry.read();
          if (c == '\n') {
            line[count] = 0;          // terminate string
            flash_hex_line(line);
            count = 0;
            my_line_count++;
          } 
          else if (c == '\r') {
            //do nothing
          }
          else
            line[count++] = c;        // add to string
        }
        entry.close();
        line[count] = 0;          // terminate string
        String s = ":flash " + String(my_line_count);
        s.toCharArray(line, 200);
        flash_hex_line(line);
        count = 0;
      }
    }
  } // for ever

} // upgrade_firmware()
 
@gonzales, you are on the right track. Code size limit is 1/2 of flash size, but the hex file is much larger due to all of the overhead of the hex file format. If you have the hex file on your SD card, you can use the latest Flasher3c.zip and replace code to "read hex line from serial port" with your code to "read hex line from file".
 
Thanks for your answer. My problem is, that my scetch is larger then 1/2 of flash size. Is there a possibility that this restriction will be lifted in the future? I see it this way, first i load small program with hex-loader, than hex-loader load new main programm.
In your Flasher said
// TODO - instead of splitting flash in half, use whatever flash is remaining. This would allow a 64K program to flash a 192K program. This would allow the two step process of 192K, downgrade to 64K, upgrade to new 192K.
Is it really possible to do this?
 
Yes, it's possible. I did a test by replacing all instances of (FLASH_SIZE / 2) with a new macro FLASH_BUFFER_ADDR. First, I defined FLASH_BUFFER_ADDR = (FLASH_SIZE / 2), so that's the same as Flasher3c. Then I tried (FLASH_SIZE / 4) and (3 * FLASH_SIZE / 4), and both of those work fine, too. The max download is (FLASH_SIZE - FLASH_BUFFER_ADDR). I need to test a little more and then I can upload. What you will need to do is create two applications. Your "small" app will have FLASH_BUFFER_ADDR = (FLASH_SIZE / 4), and your "large" app will use (3 * FLASH_SIZE / 4). That will allow you to toggle back and forth between downloading the small and large apps. To update to a new version of your large app, download small, then new large. The next step after this would be to have the app compute FLASH_BUFFER_ADDR rather than having it be a constant.
 
Here is Flasher3d.zip. This version lets you set FLASH_BUFFER_ADDR (in file FlashT3x.h). I have only tested on T3.2 so far. The size of Flasher3d is about 42K, so I have set FLASH_BUFFER_ADDR = FLASH_SIZE/4 = 64K. Within your "large" app, set FLASH_BUFFER_ADDR to (3*FLASH_SIZE/4). This simply means that only the top 1/4 of the flash is available for download, and since Flasher3d is less than 64K, that will work. The idea is that you can toggle between Flasher3d using (FLASH_SIZE/4), which allows download of 192K, and your large app using (3*FLASH_SIZE/4), which allows you to download up to 64K.
 

Attachments

  • Flasher3d.zip
    10.2 KB · Views: 76
Last edited:
Unfortunately I didn't succeed to load big program (295 kB)

first i fogot to change FLASH_BUFFER_ADDR to (3 * FLASH_SIZE/4) to load my large application
and i got this
8188 :10FFA000DC390400EC390400FC3904000C3A04008C
8189 :10FFB000203A04002C3A04006816FF1F6C16FF1F3D
8190 :10FFC000343A0400403A04004C3A0400543A040025
8191 :10FFD000603A04003C08FF1FAC39FF1FE868EAF7ED
8192 :10FFE000C7FC93E73A7A052A00F2A18001A353F8EF
8193 :10FFF00022F000BFEBFE0100190002001900020010
8194 :020000022000DC
8195 :1000000015000200150002000D000200F8684042D1
address too large

when i set correct #define FLASH_BUFFER_ADDR (3 * FLASH_SIZE/4)
i got this

8191 :10FFD000603A04003C08FF1FAC39FF1FE868EAF7ED
8192 :10FFE000C7FC93E73A7A052A00F2A18001A353F8EF
8193 :10FFF00022F000BFEBFE0100190002001900020010
8194 :020000022000DC
8195 :1000000015000200150002000D000200F8684042D1
flash_block write error 1
8195 :1000000015000200150002000D000200F8684042D1
can't flash 16 bytes to 0

i dont understand whats wrong, can you help?
PS. With small program (Blink) worked fine.
 
Last edited:
the problem is here
flash_block (base_address + address + FLASH_BUFFER_ADDR, (char *)data, byte_count)

so first offset is 3/4 of FLASH_SIZE, i think it is wrong

1 :10000000F8FF022099010000652C03001D2C03005D
393216
2 :100010001D2C03001D2C03001D2C03001D2C0300B0
393232
3 :100020001D2C03001D2C03001D2C0300652C030058
393248

8193 :10FFF00022F000BFEBFE0100190002001900020010
524272
8194 :020000022000DC
8195 :1000000015000200150002000D000200F8684042D1
524288
flash_block write error 1
8195 :1000000015000200150002000D000200F8684042D1
can't flash 16 bytes to 0

I think it will be
flash_block (base_address + address + FLASH_SIZE - FLASH_BUFFER_ADDR, (char *)data, byte_count)

but when i tryed it, i got this
18459 :108148001016FF1F1016FF1F1816FF1F1816FF1F07
18460 :108158002016FF1F2016FF1F2816FF1F2816FF1FB7
18461 :0881680000000200FFFFFFFF11
18462 :00000001FF
done, 18462 hex lines, address range 0:48170, waiting for :flash 18462
lower FSEC (fffff9deffffffff) and upper FSEC (0382eb0447704b04) do NOT match
flash 0:48170 begins...

but my big program not started((
 
In the SMALL program source code, you should have FLASH_BUFFER_ADDR = (FLASH_SIZE / 4)
In the LARGE program source code, you should have FLASH_BUFFER_ADDR = (3 * FLASH_SIZE / 4)

Is that what you have?

Which Teensy are you targeting? 3.5 or 3.6?
 
@the_boss, yes, you could adapt this serial download/flash process to Ethernet. The same person (jonr) who wrote the original Flasher program for T3.x has also written Flasher4, for T4.x, which you can find here:

https://forum.pjrc.com/threads/29607-Over-the-air-updates?p=263696&viewfull=1#post263696

I recommend you get that working and try downloading hex files via the USB serial, and then it will become clear how you can adapt it to send new firmware to your application via Ethernet. You'll need a TCP/IP client that reads your hex files, encapsulates each line (or multiple lines) in a packet and send them to the T4.x. In the T4.x application, you'll need to receive those packets, extract the hex line(s), and then process them the same as in Flasher4. Everything you need to parse the hex lines and erase/write flash is in Flasher4. You just need to use Ethernet rather than serial.
 
In the SMALL program source code, you should have FLASH_BUFFER_ADDR = (FLASH_SIZE / 4)
In the LARGE program source code, you should have FLASH_BUFFER_ADDR = (3 * FLASH_SIZE / 4)

Is that what you have?

Which Teensy are you targeting? 3.5 or 3.6?

Now i just want to load my large program to 3.5 from your Flasher3d. FLASH_BUFFER_ADDR - is offset for addressing or availible flash size?
I think it is offset, so i tryed to set
#define FLASH_BUFFER_ADDR (FLASH_SIZE/4)
and got this

8193 :10FFF00022F000BFEBFE0100190002001900020010
8194 :020000022000DC
8195 :1000000015000200150002000D000200F8684042D1
address too large

so the problem is here
if (base_address + address + byte_count > FLASH_BUFFER_ADDR - RESERVE_FLASH) {
Serial.printf("address too large\n");
error = 2;
return -4;
}

I write to Serial FLASH_BUFFER_ADDR - RESERVE_FLASH, it is 131072, that is right (524288/4=131072)
but my lagre program have 18462 strokes x 16 bytes = 295392, so this condition (if (base_address + address + byte_count > FLASH_BUFFER_ADDR - RESERVE_FLASH)) will not be executed.

my research showed, that the condition must be
if (base_address + address + byte_count > FLASH_SIZE - FLASH_BUFFER_ADDR - RESERVE_FLASH) {
Serial.printf("address too large\n");
error = 2;
return -4;
}

now i can successfully load my large program.
Thanks a lot for your work. Now i will update my large program!
 
@gonzales, yes, you're right. I simply replaced FLASH_SIZE/2 with FLASH_BUF_ADDR everywhere. In that case, the original code should have been (FLASH_SIZE - FLASH_SIZE/2 - RESERVE_FLASH) and the new code (FLASH_SIZE - FLASH_BUF_ADDR - RESERVE_FLASH).

Let us know if it works correctly for you and I will upload another ZIP file with the correction.
 
Back
Top