TFTP Bootloader, update firmware over ethernet?

thinkpeace

Active member
Are there any plans of updating the bootloader so that the Teensy can upload new programs over Ethernet?

Here are some bootloaders I found for the Arduino that provide this feature.
http://playground.arduino.cc/Code/TFTPBootloader1
https://github.com/freetronics/arduino-tftpboot
https://github.com/esmil/arduino-netboot

The Moteino also has a similar feature for updating firmware over an RF network.
http://lowpowerlab.com/blog/2013/05/05/wireless-programming-update/

This feature would be very useful if available.

Eric
 
So, this topic seems to come up again and again. Lets make it work!

Paul, what would need to get this started - I am happy to contribute time and effort (I dont have huge programming skills, but can architect, design use cases etc and test).

My envisaged solution would be (and its open for discussion)
1. External flash (SST 2mbit springs to mind) connected via SPI (if you need to do OTA updates, wired or wireless, then you probably have an advanced project that has some storage needs :cool:)
2. Application code to "download" the new hex image during the course of normal operations and push it into a defined space in the flash, in blocks at an appropriate rate
2a. Send in smallish blocks with checksum etc
2b. set a flag when received and checksums are valid
3. Reboot
3a. boot loader now checks if USB upload is ready/required (i.e. plugged into pc and teensy loader active)
3b. If not check "new code" flag in flash and copy to progflash
3c. Jump to new code when done
3d. if new code starts successfully set another flag in ext flash to ignore updating again after next reboot
4. Optionally before reboot, copy existing prig flash to another ext flash space to do a auto recovery if the new code fails to run (will involve some watchdog use though I guess)
I realise above is simple, however I helped created the above for the pic32 environment and I would love to move it to Teensy3.1 but just don't have enough skill .


Any takers ?
 
I too have a downloader for remote reprogramming. Working now with AVRs on wired and wireless (low power wireless, with low cost radios). It boots using a new-code image stored in a flash chip. That was stored by a program that receives and stores and CRCs the image. The bootloader reads the SPI flash and writes the new code to the CPU's program memory. The medium used to get the file and store it in SPI flash isn't important nor does it affect the bootloader.

The SPI chip is a 16MByte 8 pin chip with optional passworded access controls built into the chip, along with read-only locks and so on.

As I understand, the Teensy 3 boot process is for the mini54 chip to erase ALL of the flash in the K20, including where a bootloader might have been stored. I read that the mini54 chip then pushes a bootloader into the erased K20 and runs that code. That code then uses USB to talk to the Teensy loader program on the computer.

Given there's no place to install a different bootloader that won't be erased, it seems that PJRC needs a rationale for working with remote reprogramming. And currently, that's not on the radar per prior postings.

My work on this is for use in wireless sensors/controls where remote devices use $5 wireless radios. Some hobby use, but mostly for HVAC and security systems. The devices are numerous and in hard to get to places, hence the need for remote reprogramming. The same mechanism is used to pull data logs from data acquisition modes of these devices - environmental and security monitoring and logging.
 
I'm speculating here but my guess is PJRC has little/no interest in something like this because it would open the doors to Teensy 3.x clones so I doubt Paul will be very helpful in this project.

I looked at something similar last year but instead of self-programming I used an ATMega to sort of duplicate the Mini54's functionality. New code would go into a Flash chip as you describe, then the ATMega would take over and reprogram the MK20 via EZPort. Early tests worked but the additional components ended up pushing the cost too high (cost for additional components, PCB real estate, and per-board assembly), and there were weird intermittent problems getting the AtMega (SPI reprogramming) and Mini54 (USB reprogramming) to always play nice with each other so I've abandoned that approach.

The Fadecandy project did start out with a generic Teensy 3 and evolved into using a Teensy-ish clone using a USB bootloader (https://github.com/scanlime/fadecandy/tree/master/bootloader) so you might find some helpful code there, but I don't think you'll be able to use Teensyduino any more as it assumes it owns the entire memory space.

A much easier path to remote upgradeability might be to use something designed for OTA upgrades from the start, like the Spark Core.
 
I did TFTP remote reprogramming for an ARM7. I had to dump it and rewrite it for passive mode FTP - because of firewall policy at the sites where my stuff went - no TFTP allowed outside the LAN. No active mode FTP. So I implemented FTP passive mode to a server managed by me and with an ACL. But this is where IP networks (cellular, wired) were used. Some nodes were in other hemispheres, on cellular. Lesson learned: Don't use FTP or SSL FTP due to security policies. Of course, my code implemented an unattended FTP client pull, passive mode which avoiding firewall NAT issues. But I'll not use FTP for the future due to these issues.

Currently using mega328P boards, and some Teensy 3's in a network for wireless sensors, etc. as in prior posts. OTA reprogramming and two-way bulk data exchanges. (Teensy 3 has to be reprogrammed via a person and USB). The other nodes are done OTA, and are Anarduino mini-wireless boards (amazing value). I could do the same OTA on the Teensy 2 with the same mega32P bootloader I have (it does the SPI flash read). I do so wish the Teensy 3's could be OTA reprogrammed. These things are installed in nasty places where you don't want to ask people to go, ladders and all.
 
Last edited:
I really can't see why this facility would not be a massive boost for using teensy? It would then compete and IMHO be better than spark core! Paul would still sell MINI54 just with a better bootloader than allows OTA. Am I missing something fundamental?
 
Yes, I agree. Help move on from LEDs and sound to HVAC and the like where thousands of smaller businesses (vs. Honeywell et al) sell quite a bit. Mainly in low cost wireless because of the high labor costs of running cables from a controller to where the HVAC heat pumps are located. Even in hotel retrofits.
I suppose this is life on the edge of an avocation v.s. a focus on gross margin/income.
Anti-counterfeit can be done many ways - e.g., the SPI flash has the password protected data access.
 
I don't see how this could possibly be a threat to PJRC, but I do see how it could become a huge time sink. I'm already terribly over-committed on a ton of other stuff, so I'm reluctant to do much on a TFTP loader.

Perhaps a future version of the Mini54 could look for a SPI flash chip, and if present, and if a certain signature and checksum are found, it would reprogram the MK20 with the firmware image in the SPI flash. I might be willing to do that part, but you'll still have to build the application which receives data and writes it into the SPI flash. A realistic time frame for the SPI flash support might be December or early next year.

Another concern is having the Mini54 fiddle with I/O pins, looking for the SPI flash, but perhaps interfering with unknown hardware that might be hooked up to those pins. If I implement this feature (which is a pretty big "if"), it will not be the default behavior. I'll probably requires 4 bytes in the program-once memory to be written to a specific value. Of course, if I do all this (still a pretty big "if"), I'll publish an example program that writes those bytes. Once written, the Mini54 would forever look for a SPI flash on every entry into bootloader mode.

Again, I want to emphasize this isn't a certain thing, and even if I do implement this, the time frame will likely end up early next year. Something that will help sway my opinion is a TFTP or FTP or other network program that actually receives a firmware image and writes it into the SPI flash chip. Without that, it's very unlikely I would even begin on this feature.
 
Perhaps a future version of the Mini54 could look for a SPI flash chip, and if present, and if a certain signature and checksum are found, it would reprogram the MK20 with the firmware image in the SPI flash. I might be willing to do that part, but you'll still have to build the application which receives data and writes it into the SPI flash. A realistic time frame for the SPI flash support might be December or early next year.

I and others already have the code to pull down data files with strict CRC16 checking, store in SPI flash, read back to verify CRC, save metadata prepended to the data (CRC, size, name, etc). In my case, I use my messaging with RadioHead (not TFTP, FTP, or IP) and any of its many supported radios. The protocol messaging I have now is robust for error detection/correction, with a per-packet CRC and and a data set level CRC. It can use any of RadioHead's protocols and topologies: Star, static routes, mesh, all with reliable datagrams. The current bootloader looks at the SPI flash to see if there's valid metadata and data and if so, it re-programs the program memory from SPI, then overwrites the 1's in two bytes of the metadata thereby marking it as "used". The data length and data per se are not erased until a new download happens. This is to be extended to more than one storage area as the SPI flash is large.

The bootloader is now SPI chip specific in terms of its sectorizatoin (255 sectors, 64KB each), based on the chip manufacturer number read from the chip in a vendor-independent manner - JDEC standard to discover chip's memory map. To keep the bootloader for SPI and serial to 1KB of code, the chip type is assumed, not discovered, in the current bootloader.
The bootloader looks at cause of reset: power-up or reset button vs. watchdog timeout. Only the latter triggers a reprogram. There are other such strategies.

SPI chip select - I'd vote for a jumper with 2 or three choices; default pin 10.

Happy do video-demo the existing software to do all this. PC side is Python GUI and RadioHead datagram in python - used to talk to a wireless gateway (Teensy or mega328 w/any radiohead radio. Tested on Linux, Windows, Raspberry Pi (I am seeking a Mac to test on OS X). On the device side, a structured message processor and SPI flash writer, in C/C++.
 
Last edited:
I don't see how this could possibly be a threat to PJRC.

My thought was that if you published a bootloader that could copy from Flash someone could come out with a Teensy 3.x clone without a Mini54 and you'd lose sales. If that's not a concern then it's certainly OK with me!

Perhaps a future version of the Mini54 could look for a SPI flash chip, and if present, and if a certain signature and checksum are found, it would reprogram the MK20 with the firmware image in the SPI flash. I might be willing to do that part, but you'll still have to build the application which receives data and writes it into the SPI flash. A realistic time frame for the SPI flash support might be December or early next year.

I would certainly be willing to help with this. I've abandoned work on the CC3000 but if the CC3100 is out by then I will work on the interface code as I was planning on doing that anyway.

My design isn't going to use TFTP per stevech's example, I hope to have a web page where a user picks a file and uploads (similar to the way you upgrade a consumer router), but TFTP should be easy enough to add, and as said earlier the exact mechanism by which the new image gets from the user into the Flash chip shouldn't be only one specific thing.

Another concern is having the Mini54 fiddle with I/O pins, looking for the SPI flash, but perhaps interfering with unknown hardware that might be hooked up to those pins.

Would you revise the hardware so the Mini54 has direct SPI access to the Flash chip, or would it go through the MK20?

Per stevech's other email I think there should be some way to specify the CS pin. Maybe even also specify the MISO/MOSI/SCK pins for those who remap them (e.g. for the audio board).

Again, I want to emphasize this isn't a certain thing, and even if I do implement this, the time frame will likely end up early next year. Something that will help sway my opinion is a TFTP or FTP or other network program that actually receives a firmware image and writes it into the SPI flash chip. Without that, it's very unlikely I would even begin on this feature.

Like I said I've dropped developing for the CC3000 but as soon as I can get my hands on a CC3100 (mid September / early October?) I'll start work on this.
 
Very interesting concept. My recommendation would be to only warranty operation for a very select number of external SPI chips and that an outside electrical signal would trigger the upload.

Ideally, to prevent false triggers, one of the unused mini54 pins would have to be pulled high or low to enable the external upload in the first place. This would make false triggers less likely.

You guys will forget more about this stuff than I'll ever know but I'd consider breakout pins being available on the bat54 either as a roll your own option or something for a 3++ iteration. Unless there is a huge demonstrated demand, I'd only offer it on a roll your own basis since the need for this feature is likely quite limited.
 
Last edited:
How about this --

When the CC3100MOD becomes available I'll make a reference board design with an MK20DX256, Mini54, CC3100MOD, Flash SPI chip, and support circuitry, then work on the code for an upload procedure.

Once it's working I'll send one to Paul and when he has time he can make the Mini54 modifications and we can vote etc. on the upload file format (e.g. header, CRC format).

If it all works out then maybe we can judge demand and figure out if this is something that belongs on (A) Osh Park, (B) Tindie, (C) Kickstarter, or (D) pjrc.com/Sparkfun/Adafruit/etc.

(edit-it's 'CC3100MOD' I want, not 'CC3100')
 
Last edited:
I for one don't want broadband wireless. For reasons of range, cost, power and lack of need.

That shouldn't be a problem. We'd all agree on the protocol, then you could make your own designs using the reference Flash chip(s), the modified Mini54, and the wireless connectivity of your choice. Here's the way I see it working:

1. The required bytes would be written to program-once memory, to tell the Mini54 to start looking for images on a Flash chip. Maybe the pattern could include which pins to use for Chip Select and SPI, for more flexibility.
2. We would crowd-source the library for writing Flash images, e.g. password protection, checksums, versioning, etc.
3. The program would download a new image using your hardware's interface, and use the library to write it to Flash. I'd write the support for WiFi on a CC3100, someone else could write one for Bluetooth, you could write yours for your radios, etc.
4. The program would call _reboot_Teensyduino_()
5. The Mini54 would take over and see the program-once memory settings to indicate boot-from-flash then load a config page from Flash and check it to see if a new image needs to be programmed. If so, it would reprogram the MK20 with the Flash image
6. The Mini54 would reset the MK20 and the new program would start

What do you think?
 
Re the list above.. that's similar to what is done in other circles.
For Arduino/AVR, LowPowerWireless (Felix) has an Optiboot variant for remote reprogramming. What I've done in conjunction with a different vendor (Anarduino) is similar - SPI flash retrieval and reprogramming. But for the downloads of new code, this is not a bootloader issue. The bootloader doesn't care how the code got downloaded.
Indeed, I'd like to see a no-SPI flash bootloader that does an in-place reprogramming - I did that last year. It has fail-safe techniques to deal with restarting after a disrupted transfer, etc. But I don't know if the K20 allows one section of flash to write another section - which the ARM7 I used did permit. That's how I could reprogram the 128K ARM7 without needing intermediate storage in SPI flash chip or an SD card.

Alas, the Teensy 3 cannot yet play in this remote reprogramming work.

I use open source RadioHead libraries that run on AVR and Teensy 2,3 and others like Anarduino Miniwireless boards (look at those and their low cost, and relatively high volume for real world sensors and controls).

The concept I have with RadioHead is that it supports many kinds of radios as plug and play. The application layer sees the same API no matter the radio. And I gave high importance to being able to choose between topologies according to need: start, static routes with routing nodes/end nodes, and self-forming/self-healing mesh. With $5 radios that are sub-GHz, unlicensed, and thus longer range.
So I have a set of messaging methods working. They will work in any topology.

The next step up from the above is using IEEE 802.15.4 radios like Xbee Series 1 (not 2), and all of its many competitors that all interoperate via the the 802.15.4 standard. One could use Zigbee but not me, for reasons of licensing and the onerous membership in the Zigbee alliance (Zigbee is not a synonym for 802.15.4, like TCP/IP is not a synonym for 802.3 ethernet). I worked in 802.15.4 for years and know it well. I got on the RadioHead software track because of low cost radios, open source software, and that 802.15.4 radios are $20 and up as modules that are FCC certified.

So let's chat about the overall world of wireless sensors/controls, embedded, and where WiFi fits in this; it has a tiny space due to being 20MHz wide and thus short range, high cost, high power consumption, etc.
This is a broader topic that this forum. Teensy 3 would he a great platform for alternative to the use of AVRs where apps are bound by 2K of RAM and 8 bit MCUs. Some like LowPowerWireless have moved to mega1280 and so on, but that's a baby step we should skip with todays' ARM Cortex M3, M3 being so superior, but unfamiliar to those accustomed to AVRs.

I think I'm talking too much here. It's Paul's business decision on enabling non-counterfeitable remote reprogramming. It's doable.
 
Last edited:
Interesting. That's how I did the flash on the fly :)eek:) reprogramming without intermediate bulk storage.
The problem: 100+ units in the field. Need to update firmware remotely via Internet. No flash card, no big RAM, no where to store entire flash image. CPU was ARM7 with 128KB flash. Application size about 100K.
So the secondary boot loader (primary remained UART) - had a passive mode FTP client protocol I wrote. It used a WizNet NIC. I wrote the secondary bootloader. Primary was built-in (NXP).
Connected and pulled down .hex text form. Converted to binary upon receipt. Store binary form in RAM, up to one flash sector size. That was 8KB. Repeat until end of file. Then write some bytes at an absolute address (in the linker file) to tell the bootloader that the new code is runnable.
At reset, power failure, loss of connection to internet, etc., secondary loader got control, and if the new code was NOT marked OK (CRC) it reloaded all sectors.
This worked well. It had to. Else somone (?) had to get on a plane trip for a day and a half. No one on site to help.
Lots of use of watchdog timer.
We did lots and lots of firmware updates to these units that had no bulk storage. It was a pull, not a push. So we just put new firmware on the server and short time later, all nodes were updated automatically.
The flash chip is a real luxury, even if it's only $1 for the initial design. As a retrofit, it was faster/cheaper/easier to do the flash-on-the-fly. I was motivated to get all bases covered to avoid airplanes to bad places.

So this applies as a possiblity for products like Teensy 3 where for reasons or cost or board space you don't want to add a flash chip. And an SD card/socket is too vulnerable to dirt/damage unless inside a NEMA box.
 
Last edited:
Hi All

I am using remote Ethernet updates via Web server, TFTP and/or FTP but this needs an external Ethernet controller connected via SPI (or parallel port) when the Kinetis doesn't have inbuilt Ethernet.

The Ethernet controller that I have integrated is the ENC424J600 (10M/100M) although I have also started work with the Nordic nRF24L01 (costs about $2) for the same in low power radio networks - this is a bit trickier because it carries only small frames and so needs additional fragmentation/defragmentation techniques.

Practially I tend to use the Web server mode since it only needs about 16..18k of code space (depending on compiler) since it is very user friendly (TFTP is a nuisance to set up a server for it to work with so is not practical for many people just wanting to connect from their PC or smart phone etc. and using just UDP and TFTP doesn't save a great deal of code space [for modern chips]).

The web server based updates are an option in the present uTasker serial loader and the uTasker project but the ENC424J600 interface is not yet released there. The serial loader will however work with the TEENSY and Teensy 3.1 Arduino/Teensyduino applications. The uTasker application can in addition use the uTasker "BM" (bare minimum) loader which is a 2k sized loader that will also work with these techniques, but requires the originally loaded application to actually do the data transfer to intermediate storage space.

Regards

Mark
 
Wow guys, this is a great thread. Thank you all for your time an effort on creating a solution. My biggest interest is reprogramming the teensy 3.1 via bluetooth using the Arduino IDE. Unfortunately so much of what is discussed above is way over my head. I am a very novice programmer (trained in fine art rather than CS or MechE) and am just learning how deep this rabbit hole really goes. If there's anything I can do to support this project please let me know. I do know that if I could reprogram my teensies over bluetooth I would continue to buy more and more of them. This is a very valuable feature.

Thanks again,
Chris
 
Back
Top