EHEX has EOF twice in the file - Confuses FlasherX

D

Dogbone06

Well-known member
I've compiled using PlatformIO.

I noticed problems when using FlasherX that it wouldn't move the data. Turns out that EHEX has two EOF's for some strange reason.
At the end we have the regular EOF but about 200 lines further up there is another EOF. This confuses FlasherX.

Should it really be like this?

1708188568942.png
 
I read the section on EHEX files a couple of times. It's pretty amazing that Paul provides all of this information, and I hope he won't mind answering another question.

With a standard HEX file, FlasherX writes the new firmware to the flash base address (6000 0000) and then simply does a software reset, so the processor boots and runs the new firmware from the flash base address.

With an EHEX file, the second application in the file is described as a loader utility that runs from RAM, so I'm wondering if the following would work:
  • write the main application to flash base address as usual
  • write the loader utility to RAM at the specified address (2020 8000)
  • jump to the loader utility (instead of doing a software reset)
Paul, can you tell us if that would work, and is roughly equivalent to what the Teensy bootloader does when it receives an EHEX file?

I think that would mean the loader utility is only required to boot a new program for the first time, and somehow after that, the loader utility is not required, and the processor can boot "normally", i.e. from the code at the flash base.
 
joepasquariello said:
I read the section on EHEX files a couple of times. It's pretty amazing that Paul provides all of this information, and I hope he won't mind answering another question.
...
Click to expand...
I scanned that partially once following the link - it has expanded since I last looked at it.

So, without regard to much of that ...
I've built 3 sketches that upload to a LOCKED T_4.1. :: 02:51:42.798 (loader): using encrypted ehex (required - secure mode is locked)
> simple 'Hello" to USB
> example EEPROM Read
> SD/Listfiles from SD card that is larger at 177 KB where two above are both 67 KB
Used Notepad on a copy of the .eHex and saved off after the first EOF.

Those three files have a (smallish) section of mismatches in file compare - so they are not always having the exact content - but do in the majority.

Then the content after the first EOF from the three sketches is overwritten onto one of the other sketches replacing all after the first EOF.
Those edited .eHex files still work for Teensy Loader to upload and have the Locked T_4.1 come up running.

Not sure why the post EOF#1 has changed bytes - but doesn't seem to affect the process that code uses to allow the MCU HAB process to complete.

And Super Odd / Unexpected - deleting everything after the first EOF#1 and saving that as the .eHex also Uploads and executes perfectly fine when the 14KB of post EOF#1 data is removed.
 
defragster said:
And Super Odd / Unexpected - deleting everything after the first EOF#1 and saving that as the .eHex also Uploads and executes perfectly fine when the 14KB of post EOF#1 data is removed.
Click to expand...

I believe you have found a previously unknown bug in Teensy Loader. If the 2nd part of .ehex is missing (which never happens when written by teensy_secure), but you previously loaded a good 2-part .ehex file, Teensy Loader ignores the missing-2nd-part error and reuses the previously loaded 2nd part which is still sitting in its memory.

This bug isn't likely to be fixed until a full rewrite of Teensy Loader. Sadly, the existing code was designed (15 years ago) around the concept of a single 1-part hex file. This .ehex stuff was bolted on (about 2 years ago). Loading .ehex is actually handled as 2 different cases, either loading .hex and .ehex is found alongside it, or explictly loading .ehex without .hex. Neither case has all possible .ehex errors properly rejected. Some checking of the .ehex data is done, but it's focused on verifying the Teensy you're about to program was set up properly. The .ehex file isn't checked as well as it should be, and because the old code structure is designed around a single 1-part .hex file, that sort of more rigorous checking probably won't ever be added to the existing Teensy Loader.
 
PaulStoffregen said:
I believe you have found a previously unknown bug in Teensy Loader.
Click to expand...
Confirmed. Even with Verify the Teensy.exe grabs/reuses .eHex part #2 when it is missing.

But Close and Open Teensy.exe and point to an .eHex with No part #2 and:
1708281889907.png


Still interesting that .eHex part #2 is marginally unique - but any .eHex part #2 seems to work for any sketch that requires it.
 
joepasquariello said:
I read the section on EHEX files a couple of times. It's pretty amazing that Paul provides all of this information, and I hope he won't mind answering another question.

With a standard HEX file, FlasherX writes the new firmware to the flash base address (6000 0000) and then simply does a software reset, so the processor boots and runs the new firmware from the flash base address.

With an EHEX file, the second application in the file is described as a loader utility that runs from RAM, so I'm wondering if the following would work:
  • write the main application to flash base address as usual
  • write the loader utility to RAM at the specified address (2020 8000)
  • jump to the loader utility (instead of doing a software reset)
Paul, can you tell us if that would work, and is roughly equivalent to what the Teensy bootloader does when it receives an EHEX file?

I think that would mean the loader utility is only required to boot a new program for the first time, and somehow after that, the loader utility is not required, and the processor can boot "normally", i.e. from the code at the flash base.
Click to expand...

@PaulStoffregen, I would very much appreciate if you can reply to the questions that @joepasquariello wrote above.
I suspect there may be NDA stuff prohibiting you from doing that, if that's the case then please let me know. Thanks! 🙏
 
Confirm, I generally don't comment on this code security stuff beyond the info already published on the code security web page and in the core library code, the 3 automatically generated programs and Arduino IDE plugin code. The NDA and uncertainty about which specific details the NDA really covers is the main reason.

We spent months in emails with people at NXP. Clear definitive answers always seemed elusive. After much back and forth, I sent NXP a draft of the web page and screenshots of the plugin and copies of the generated code for a simple yes / no answer. Even that took quite a bit of back and forth, but finally they said yes, they gave us the green light to publish that specific set of code and documentation.

The questions I answer are pretty much limited to only that info, because people at NXP specifically reviewed that code and documentation and gave their approval for PJRC to release it.
 
Last edited:
Here's something interesting. I use FlasherX as mentioned before. I just did something dirty, I removed the second part of the EHEX EOF. Meaning, I removed the 200ish lines after the first EOF. And then had FlasherX use that EHEX to update my device. The device is of course encrypted with a custom key (new key generated in Teensy Security). And it works!! I first thought it was a fluke but I've tested again and again. It works!

Makes me wonder what the second part even does. It seems useless.
 
Dogbone06 said:
Makes me wonder what the second part even does. It seems useless.
Click to expand...
Perhaps the "loader utility" that is contained in the 2nd part of the EHEX file is not needed when the update is done via FlasherX, i.e. from within an application that has already passed the "security barrier"? I know so little about the locked teensy and security features that I'm really just guessing, but if you're right, it's good news. I hope to use locked T4.1 in an application that requires remote update.
 
joepasquariello said:
Perhaps the "loader utility" that is contained in the 2nd part of the EHEX file is not needed when the update is done via FlasherX, i.e. from within an application that has already passed the "security barrier"? I know so little about the locked teensy and security features that I'm really just guessing, but if you're right, it's good news. I hope to use locked T4.1 in an application that requires remote update.
Click to expand...
I guess we'll never truly know as this is secret stuff. I am just happy that FlasherX can use the EHEX when removing the last part. I use FlasherX for everything these days, it's so conveniant and safe to have the device update itself. Using the ordinary way, a bad USB cable can brick the device by interupting the communication. This simply cannot happen with FlasherX.

I am forever greatful that you made it Joe! 👏
 
Dogbone06 said:
Using the ordinary way, a bad USB cable can brick the device by interupting the communication.
Click to expand...

This is not true. Lockable Teensy (when permanently set to secure mode) will reappear after power cycling in USB bootloader mode if the flash memory gets corrupted, either by a failed upload or any other cause that alters even 1 byte of the encrypted portion.

The hardware does not become bricked (which everyone understands "bricked" means an unrecoverable state which essentially turns powerful hardware into a brick). Lockable Teensy in secure mode starts up in bootloader mode if the flash memory doesn't contain a properly encrypted program. To recover, all you need to do is connect a good USB cable and successfully complete an upload so the flash memory has a full properly encrypted program.
 
PaulStoffregen said:
This is not true. Lockable Teensy (when permanently set to secure mode) will reappear after power cycling in USB bootloader mode if the flash memory gets corrupted, either by a failed upload or any other cause that alters even 1 byte of the encrypted portion.

The hardware does not become bricked (which everyone understands "bricked" means an unrecoverable state which essentially turns powerful hardware into a brick). Lockable Teensy in secure mode starts up in bootloader mode if the flash memory doesn't contain a properly encrypted program. To recover, all you need to do is connect a good USB cable and successfully complete an upload so the flash memory has a full properly encrypted program.
Click to expand...
That's nice to know, I didn't know that. Haven't used any locked Teensy for real world use yet. I just tested it and you are right, a "bricked" (locked) Teensy will in fact reappear and can successfully be flashed. This is very good!

But for an unlocked Teensy it get's "bricked" if I upload code and jank the USB cable in the middle of the update. Atleast that's what I've seen before.

Some more info on the FlasherX EHEX upload is that I found one issue.
My device has a battery, it works fine if I first upload with PlatformIO, I can then use FlasherX as many times as I want. But if I reboot the device (shut off for real, killing the power to the board). Then FlasherX wont work anymore. Device get's "bricked". So that second part do play some role, I wish we knew what.
 
Last edited:
You must log in or register to reply here.
Back
Top