OTA through Ethernet with Teensy 4.1

AndyA, are you saying that including flasherX in my code should be enough to meet this check?
Well that didn't seem to work for me. I had to use it somewhere in the code before it would work.
Although that makes sense I'll check it again.
 
You shouldn't need to manually include it. I never did. For the code to check whether a string is there it needs to include that string so it knowns what to look for. It's a self fulfilling requirement which does make it a very smart way to implement a basic sanity check that code has been built for the correct target before installing it.
I'm not sure why it didn't work for you, I suppose the compiler could have been too clever when it comes to optimising things but it does seem odd.

If you do need to manually include something then rather than having a Serial.print("fw_teensy41"); in your code I'd recommend using Serial.print(FLASH_ID); that way you'll always include the correct string that it is looking for based on the device you're compiling for.
 
I'm working on using this flasherX to remote update over Ethernet. I got the transfer of the hex file to the teensy4.1 working, but I not sure how to get the lines of hex into the buffer. Does the (flash_write_block) keep track of the address to write the lines to or do I need to do that?
I looked at the async web server example but I dont want to use that code in the final write.
Thought about saving the hex lines to the sd card and them using the sd example in flasherX to finish.
Any Ideas?
 
The buffer must contain the binary image, not the hex lines. Since the hex lines are being sent via Ethernet, you can either convert to binary as the packets are received, similar to receiving via UART, or you can write the hex lines to SD and then read the file and convert to binary. Either one will work. I don't know how the async web server works, so I don't know whether one would be simpler/easier. If the async web server directly supports file transfer, then it would probably be easier to think of the process as transferring the hex file to SD, then updating from SD.
 
Define a variable to track the hex decode status
C: 
hex_info_t hex = {
    // intel hex info struct
    data, 0, 0, 0,     //   data,addr,num,code
    0, 0xFFFFFFFF, 0,  //   base,min,max,
    0, 0               //   eof,lines
  };

Once you have received the HEX file data in a loop split the received hex data into text lines based on 0x0a or 0x0d characters
Filter out any lines with 0 length
Then call function below with a pointer to the start of each line in turn.
If it returns false at any point abort the upgrade.
C: 
bool processHexLine(const char* line) {
  if (parse_hex_line(line, hex.data, &hex.addr, &hex.num, &hex.code) == 0) {
    Serial.printf("abort - bad hex line %s\n", line);
    return false;
   } else if (process_hex_record(&hex) != 0) {  // error on bad hex code
     Serial.printf("abort - invalid hex code %d\n", hex.code);
    return false;
   } else if (hex.code == 0) {  // if data record decoded OK
      uint32_t addr = buffer_addr + hex.base + hex.addr - FLASH_BASE_ADDR;
      if (hex.max > (FLASH_BASE_ADDR + buffer_size)) {
        Serial.printf("abort - max address %08lX too large\n", hex.max);
        return false;
      } else if (!IN_FLASH(buffer_addr)) { // copy to buffer in ram
        memcpy((void*)addr, (void*)hex.data, hex.num);
      } else if (IN_FLASH(buffer_addr)) {  // copy to buffer in flash
        int error = flash_write_block(addr, hex.data, hex.num);
        if (error) {
          Serial.printf("abort - error %02X in flash_write_block()\n", error);
          return false;
        }
      }
    }
    hex.lines++;
  }
  return true;
}

Once whole input has been processed you will have the binary firmware image in the flasherX buffer.
You can then run a basic check on it and if that passes install it using flash_move.
C: 
if (check_flash_id(buffer_addr, hex.max - hex.min))
    flash_move(FLASH_BASE_ADDR, buffer_addr, hex.max - hex.min);

In addition to the flashID check that flasherX uses I also run my own sanity check that the image has the correct header.
This is teensy4 / 4.1 only:
C: 
bool VerifyValidImage(uint8_t *imageStart) {
  uint32_t SPIFlashConfigMagicWord = *((uint32_t*)imageStart);
  uint32_t VectorTableMagicWord = *((uint32_t*)(imageStart+0x1000));
  if ((SPIFlashConfigMagicWord == 0x42464346) && (VectorTableMagicWord==0x432000D1)) {
    uint32_t imageEntryPoint = *((uint32_t*)(imageStart+0x1004));
    if ((imageEntryPoint < (FLASH_BASE_ADDR+0x1000)) || (imageEntryPoint > (FLASH_BASE_ADDR+0x2000))) {
      Serial.printf("Image indicates a code entry point at 0x%08X, this is outside the expected range.\r\n", imageEntryPoint);
      return false;
    }
    Serial.printf("Image indicates a code entry point at 0x%08X.\r\n",imageEntryPoint);
    return true;
  }
  Serial.printf("Image doesn't have expected ivt table magic number\r\n");
  return false;
}


With the exception of the extra verification step this is essentially the process both the web version and the SD card file reading are doing, the only difference is where they get the hex file from. In your case it will be a buffer in memory that was populated with data from a network socket.
 
Thanks AndyA, question the hex.data, &hex.addr, &hex.num, &hex.code are not declared, nor are process_hex_record, parse_hex_line
what library am I missing that these come from?
 
As @joepasquariello indicated they are from flasherX

I moved the lines
Code: 
typedef struct {  
  char* data;         // pointer to array allocated elsewhere
  unsigned int addr;  // address in intel hex record
  unsigned int code;  // intel hex record type (0=data, etc.)
  unsigned int num;   // number of data bytes in intel hex record

  uint32_t base;  // base address to be added to intel hex 16-bit addr
  uint32_t min;   // min address in hex file
  uint32_t max;   // max address in hex file

  int eof;    // set true on intel hex EOF (code = 1)
  int lines;  // number of hex records received
} hex_info_t;
int parse_hex_line(const char* theline, char* bytes, unsigned int* addr, unsigned int* num, unsigned int* code);

Into fxutil.h and then #included that in my code.
hex is then a variable of type hex_info_t.
Sorry I'd forgotten I'd made that change.
 
Hi. I am trying to load the hex file on the teensy 4.1 with the demo code. I get this report back :

15:39:06.840 -> Starting OTA...
15:39:06.840 -> created buffer = 0K FLASH (607FC000 - 607FC000)
15:39:06.840 -> abort - max address 60000010 too large

What am I doing wrong ?
 
Are you using EEPROM and/or LittleFS?

In Flashtxx.h, in the section for TEENSY41, if the FLASH_RESERVE is 4 sectors, increase it to 64 sectors as shown below.

Code: 
  #define FLASH_RESERVE         (64*FLASH_SECTOR_SIZE)   // reserve top of flash
 
Eeprom or LitleFS ? Maybe I get it wrong. I put the demo code on the teensy 4.1 thru usb and thru webinterface over ethernet I am trying to upload the hex file. So it should replace the original code. At least that ist my understanding.
 
The Flasherx library needs a place to put a temporary copy of the new code as it receives it. Once it's received the whole thing it does a basic sanity check on it and then replaces the current code with the new one. This way if something goes wrong during the upload it doesn't break things.

FlasherX defaults to putting this buffer in flash, it works out where is safe to put it by starting at the highest address and then looking backwards until it finds a part of flash with data in. It then assumes anything after that last bit of data is OK to use.
If you are using (or have in the past used) the eeprom or littleFS libraries then these use that top area of flash to store their data. The flasherX library sees that and thinks the flash is full up and doesn't have space for the buffer.

The change @joepasquariello gave you tells the flasher library to skip that very top area of the flash that the other libraries use and put the buffer at a slightly lower address in the flash.
 
Thank you for your help. I have erased the memory with the push button. When I try to reflash the teensy_ ota demo code it looks like it is working. So I made just a simple code that sends over Serial print update successfull. When uploading I get report about missing string "fw_teensy41" so there needs to be the flasher library anyway in the code ?
 
As a simple way to verify that you are uploading the correct firmware the library looks for the board version string to be somewhere in the uploaded file, "fw_teensy41" in your case. This stops you accidentally trying to load teensy3 firmware onto a teensy 4.
Since the library includes this string in order to check for it this condition is automatically met if the new firmware you are loading also includes the flasherx library. This is a reasonable assumption since it's rare for someone to want to update firmware to a new version that doesn't have the ability to update the firmware.

If you don't want the new firmware to include the flasherX library then simply ensure that string is somewhere in your code. Or modify the library to skip the check or look for a different string that is in your code.
 
I just commented out that part in the demo code about the string and it looks like it is working now. I know it is not a solution it is just one time flash over the ethernet, but now I can see that it works. Dont understand how exactly should be putted the string "fw_teensy41" in the code. Tried the thing to define the flash_ID as it is mentioned earlier in the comments about the sdcard but did not worked for me.
 
A few things would work.
The nice way is to define a const char[] with that value and then make sure your code uses it so that the compiler doesn't optimise it out.
Or the most basic solution would be to including the line
Serial.print("fw_teensy41");
somewhere in your setup() code.

If you look a few posts earlier I posed a VerifyValidImage() function that can be used as an alternative way of performing a basic sanity check on the uploaded code either in addition to or instead of the string check. This is only valid for Teensy4.x devices and verifies that the start of the uploaded file has the expected structure. It won't catch a 4.0 image being loaded onto a 4.1 board but is far better than nothing and means that a bit of code that just happens to list all the teensy variants won't automatically pass the tests.
 
Is there no simple working example for this library? OK I'm not very clever, but I would like to have a sketch to install on the master Teensy and a sketch to install on the slave Teensy and a script or whatever to run on my PC that picks up a compiled hex file from the PC, passes it to the master, the master passes it to the slave and the slave installs the new hex file and reboots. FlasherX.ino is a single sketch. Surely there needs to be a master sketch and a slave sketch, both or which also include so 'normal' stuff like blinking leds or sending Midi?
FlasherX.ino seems to be the sketch that goes in the slave, but where does MY actual sketch go... not inside FlasherX.ino's void loop() because that seems busy moving data around and rebooting. So if it's not an example of the slave sketch, are there examples somewhere that I've missed that operate in the required master and slave roles? If someone has got this to work and is able to have something like a slave sketch flashing a led at 1Hz and to send a new sketch to it that flashes at 2Hz, then please share the sketches because I really don't understand what FlasherX does! Thanks.
 
The teensy compiler creates a .hex file. The teensy loader normally then uploads this over USB. Flasher X provides a way for your program to take the same hex .file and install it without needing some other tool or a USB connection to a PC. Only the one teensy is involved normally, the one you want to upgrade.

FlasherX.ino should be considered an example. Like most of the examples it's not code you would normally build and run directly except to prove that the basic idea works.
Normally you would add most of this code to your main application as a function and then call that function at the appropriate time. This could be a serial command to enter upload mode, an SD card being inserted with a suitably named file or whatever other method you pick for getting the new firmware to the device and telling it to update.

Since the whole point is to allow the teensy to update its own firmware there is the assumption that the flasherX code will also be be included in the new firmware that you are uploading. Without this you wouldn't then be able to upgrade again in the future. The library checks for this and will fail if you try to install firmware that doesn't pass this test.

Some of the stuff in this thread is getting into more complex situations where you need to be able to recover a system if invalid firmware is somehow loaded or power fails in the middle of an upgrade. But those situations are for the truly paranoid, 99% of the time if something like that was to happen it's easier to plug the USB in and use teensy loader to recover.
 
sw_hunt said:
Is there no simple working example for this library? OK I'm not very clever, but I would like to have a sketch to install on the master Teensy and a sketch to install on the slave Teensy and a script or whatever to run on my PC that picks up a compiled hex file from the PC, passes it to the master, the master passes it to the slave and the slave installs the new hex file and reboots. FlasherX.ino is a single sketch. Surely there needs to be a master sketch and a slave sketch, both or which also include so 'normal' stuff like blinking leds or sending Midi?
FlasherX.ino seems to be the sketch that goes in the slave, but where does MY actual sketch go... not inside FlasherX.ino's void loop() because that seems busy moving data around and rebooting. So if it's not an example of the slave sketch, are there examples somewhere that I've missed that operate in the required master and slave roles? If someone has got this to work and is able to have something like a slave sketch flashing a led at 1Hz and to send a new sketch to it that flashes at 2Hz, then please share the sketches because I really don't understand what FlasherX does! Thanks.
Click to expand...

@AndyA provides a good answer to your question, and I can add a little more.

FlasherX can be used in many different ways. There's no way I could provide a "simple working example" for every possible usage. The core functionality of FlasherX is the low-level stuff:
  • with new firmware stored on Teensy (RAM, Flash, SD, etc.)
    • erase old firmware from boot memory
    • copy new firmware from storage location to boot memory
    • erase new firmware from storage location (if stored in flash)
    • reboot to run new firmware
What FlasherX does NOT do is specify HOW to get the new firmware into the target Teensy.

The example sketch is designed to show the low-level stuff works, and supports these simple options for getting the new firmware into the target:
  • transfer via USB or hardware serial and buffer in Flash or RAM
  • read file from built-in SD
These are not production solutions. For the USB or hardware serial transfer, the sketch reads the new firmware via a serial stream. This allows someone to send the new firmware using something like a terminal emulator that supports "file send", so that no custom client is required. The user manually confirms that the file has been transferred correctly.

In my own applications, I generally have a custom Windows client that sends the new firmware to the Teensy with a packet-based protocol. The target Teensy sketch contains code to read the packets, extract the payload and add it to the buffer. When the entire firmware has been transferred, the Windows client and Teensy do a handshake to confirm that the new firmware is correct and complete, and if so, the Windows client sends a command to tell the Teensy to go ahead with the update.

I don't know a lot about Ethernet, but I think you could do something relatively low-level, like what I do but over Ethernet instead of serial, or something higher-level such as having your master Teensy be an FTP server that can receive a file from an FTP client and write it to SD. Once you have the file on your master Teensy's SD, the master can open the file, read it block-by-block, and sending each block to the slave Teensy over serial or whatever interface you have between master and slave. The slave Teensy could have something like the FlasherX example sketch, receiving the new firmware over serial, buffering in Flash or RAM. The one thing you would have to add is a way for the slave to know that the transfer is complete, and hopefully a way to compute and test a checksum or CRC. It's pretty important for the target to confirm that the firmware is correct and complete before erasing the existing firmware and writing the new firmware to program flash.

One thing I would like to add is an example where the Teensy sends it's own firmware to itself, such as send on Serial1 and receive on Serial2, and update when complete. The "new" firmware would be the same as the old firmware, but it would show how to implement send/receive using a library like SerialTransfer, which can send/receive packets containing arbitrary data, in this case I would have each packet contain one line of the hex file.
 
Hi, how do I make this to compile for Teensy Micromod instead of Teensy 4.1?

I started with this fork and its demo sketch: https://github.com/FogozIV/FlasherX-Ethernet_Support
AsyncWebserver: https://github.com/khoih-prog/AsyncWebServer_Teensy41
QNEthernet: https://github.com/ssilverman/QNEthernet

Fork says "Over-the-air firmware updates for Teensy LC/3.x/4.x/MicroMod" but when I compile the example sketch, I only get

Documents\Arduino\libraries\Teensy41_AsyncTCP\src/Teensy41_AsyncTCP.h:61:4: error: #error Only Teensy 4.1 supported
61 | #error Only Teensy 4.1 supported

Please help, Michael
 
The async_TCP library is intended to work with the ethernet built in to the processor rather than an external SPI connected network.
You need to replace it with a version designed for the micromod / W5500.
 
Thanks AndyA, could you point me a bit more in the right direction?

Googling for "async_tcp library W5500 teensy" gives me some results but frankly I do not understand what to do with them.
 
Tip: the QNEthernet library has support for the W5500. You’ll need to change an option inside the qnethernet_opts.h file, and possibly the W5500 driver file, depending on how the hardware is connected.
 
You must log in or register to reply here.
Back
Top