Teensy 4.1: flashing fails every other time

flok

flok

Active member
Hi,

Received my Teensy 4.1 in the mail today. Looks good!

Flashing fails every 1st time though (2nd succeeds, 3d fails, 4th succeeds etc)

Code: 
Configuring upload protocol...
AVAILABLE: jlink, teensy-cli, teensy-gui
CURRENT: upload_protocol = teensy-cli
Rebooting...
Uploading .pio/build/Teensy4_1/firmware.hex
Teensy Loader, Command Line, Version 2.2
Read ".pio/build/Teensy4_1/firmware.hex": 470016 bytes, 23.1% usage
Found HalfKay Bootloader
error writing to Teensy

*** [upload] Error 1

What can it be?
 
PaulStoffregen said:
Longer timeout is needed when the flash was previously written with a large program.

Try it with this change to teensy_loader_cli

github.com

Allow longer timeout on early blocks for erase time · PaulStoffregen/teensy_loader_cli@7c75c60

Command line Teensy Loader. Contribute to PaulStoffregen/teensy_loader_cli development by creating an account on GitHub.
github.com github.com

Does this fully solve the problem you're seeing?
Click to expand...

Hi,
Thanks.
It did change the behaviour indeed. It looks like it now works altough one time I did not even pressed reset and it already started flashing (or maybe the reset button is very sensitive and I did not notice that I was touching it already).
 
Could also be a loose USB connector. Had the same happen here once, connector is loose and the PCB isn't secured to anything "heavy" like a breadboard, so pressing the buttons tends to push the board around.
 
F.w.i.w.: after, say, 50 flashes it becomes unreliable. Like not always booting up. Only reflashing it helps then, sometimes after 2 or 3 times.
 
Could you try with the normal graphical Teensy Loader? If things go wrong, it gives much more detail in Help > Verbose Information.
 
PaulStoffregen said:
Longer timeout is needed when the flash was previously written with a large program.

Try it with this change to teensy_loader_cli

github.com

Allow longer timeout on early blocks for erase time · PaulStoffregen/teensy_loader_cli@7c75c60

Command line Teensy Loader. Contribute to PaulStoffregen/teensy_loader_cli development by creating an account on GitHub.
github.com github.com

Does this fully solve the problem you're seeing?
Click to expand...
Hi Paul, are you able to make available a GUI version of Teensy loader with extended timeout for MAC OS (like the experimental version you did for Windows recently) ? It would be a great help to me. Thanks very much.
 
rvh said:
are you able to make available a GUI version of Teensy loader with extended timeout for MAC OS
Click to expand...

Yes, mostly. Here's a copy I built just now, just for you.


It's signed, but not Apple Notarized. If your Mac has default config, you might not be able to run it without editing security settings. Even then, you might need to hold a key like Shift or Command while clicking. Sorry, this is the best I can do for a quick build. The Apple Notarization stuff is a real pain. I'll do it again on the next beta.

But right now I'm also unable to sign for Windows since PJRC's code signing cert expired recently and we're waiting for a new one to be issued. Looks like this stuff is also having mandatory (forced) changes on both platforms, so I might need to rework how I build Teensyduino soon. So not what I need right now!

Hopefully you can find a way to run this not-Notarized build.
 
PaulStoffregen said:
Yes, mostly. Here's a copy I built just now, just for you.


It's signed, but not Apple Notarized. If your Mac has default config, you might not be able to run it without editing security settings. Even then, you might need to hold a key like Shift or Command while clicking. Sorry, this is the best I can do for a quick build. The Apple Notarization stuff is a real pain. I'll do it again on the next beta.

But right now I'm also unable to sign for Windows since PJRC's code signing cert expired recently and we're waiting for a new one to be issued. Looks like this stuff is also having mandatory (forced) changes on both platforms, so I might need to rework how I build Teensyduino soon. So not what I need right now!

Hopefully you can find a way to run this not-Notarized build.
Click to expand...
Thanks very much Paul, that's greatly appreciated. I won't be testing it myself as I know virtually nothing about MAC OS, but have someone else who is testing it for me. I'll pass on your comment about the notarization. Regarding the next beta, do you have any idea what sort of time scale we're talking about?
 
After further investigation (back on my own windows system), I found that if I program the T4.1 with it removed from my custom circuit board I can get it to work every time even without the extended delay. I subsequently put together a circuit board that contains everything except the ILI1394 display, and in that case the T4.1 programs correctly mounted in the circuit board too. So my problem seems somehow related to the IL1394 display or its code. I'm using the standard touchscreen library, and these are the pins and includes I'm using:

Code: 
#include <XPT2046_Touchscreen.h>

#define TFT_MOSI 11           
#define TFT_SCLK 13       
#define TFT_MISO 12       
#define TFT_SPI_RATE  30000000
#define XPT_CS 41               
#define TFT_CS 10
#define TFT_DC 9
#define TFT_LEDPIN 37             // display has transistor so can drive this from Teensy

#include <ILI9341_t3.h> 
#include <font_Arial.h>
ILI9341_t3 tft = ILI9341_t3(TFT_CS, TFT_DC, 255, TFT_MOSI, TFT_SCLK, TFT_MISO);

With minimal initialization
Code: 
tft.begin();
 tft.setRotation(0); 
 if (!ts.begin())           // maybe put an error message here
    ; 
 tft.fillScreen(ILI9341_BLACK);

I don't know what happens when I continue to make calls to the touch/display library functions when there's no display connected. But what puzzles me is that with the display present I inserted setup code before any display initialization to detect whether a button was being held down during power up, and in that case not proceed to any other code. But using that I still had the large program erase timeout error. Remember also that with the display present, the problem doesn't occur for smaller FLASH code/data.

Any suggestions as to what I might look at next would be much appreciated.
 
Last edited:
It might be not the code, but power. If you have low quality USB cable, it has significant power draw. When you are using extra devices, like display etc, the teensy gets even less current.
 
aleglakov said:
It might be not the code, but power. If you have low quality USB cable, it has significant power draw. When you are using extra devices, like display etc, the teensy gets even less current.
Click to expand...
Thanks, but my board has multiple regulators driven from a 9V plug pack, and doesn't use the USB 5V supply (track is cut). The Teensy, display, codec and analog circuits all have their own regulators.
 
No matter how many voltage regulators you're using, power problems would be my first guess too when programming (the same .hex file) works or fails depending on whether a display is connected.

Maybe I'm reading too much into your question, but just in case there's any confusion, please allow me to clear up a couple things. The process of entering bootloader mode fully resets the RT1062 chip. So the processor always starts with with a clean slate of initial reset state. But the bootloader doesn't know about external hardware you've connected, so any extra hardware continues on running as if Teensy suddenly stopped talking to it.
 
Especially if any of your voltage regulators are small DC-DC converter modules, I would recommend watching their outputs with an oscilloscope. Some of them are pretty notoriously bad. In the early days of Teensy 3.x, many people had a lot of difficult problems with a little Traco brand converter Adafruit was aggressively selling at that time. It performed horribly. Even worse, its datasheet specifically said a capacitor at the input was only needed if the input voltage was on the upper end of its range. But in fact it needed that capacitor to run reliably with almost any input (other than another power supply having a large capacitor). At the time, whenever people had power supply troubles Adafruit would recommend adding more capacitors to the output, which only made the problems even worse. In that case, the Traco's internal feedback loop was going unstable because it would try to suddenly move too much charge from input to output, causing the input voltage to drop too suddenly. What that particular Traco unit needed was more capacitance at its input, not its output (or more at both, but the input was of primary importance).

My long-winded point is to "trust by verify" your voltage regulators. Tiny DC-DC modules are the worst. Some LDO linear regulators also require specific types of capacitors and can go unstable and behave badly if the wrong type of capacitor or too much capacitance is used.

In fact, here's a scope screenshot from a recent project where 3 power supplies start up together under software control (the yellow waveform). The green waveform is a misbehaving LDO regulator with too much low ESR capacitance. Crazy thing is it's still producing approximate the right voltage and looks to be ok with small error if checked using only a DC voltmeter. But with the scope you can see its output is oscillating by about 20% of the rated output voltage! More capacitors is not always merrier!


1709904101098.png
 
Last edited:
PaulStoffregen said:
No matter how many voltage regulators you're using, power problems would be my first guess too when programming (the same .hex file) works or fails depending on whether a display is connected.

Maybe I'm reading too much into your question, but just in case there's any confusion, please allow me to clear up a couple things. The process of entering bootloader mode fully resets the RT1062 chip. So the processor always starts with with a clean slate of initial reset state. But the bootloader doesn't know about external hardware you've connected, so any extra hardware continues on running as if Teensy suddenly stopped talking to it.
Click to expand...
Thanks Paul, I appreciate the added comments. However, the Teensy in my circuit board has a well bypassed linear LM7805 T0-220 regulator all to itself so it can optionally drive a MIDI controller on the USB host port. And the touch display is powered by a separate LM7805 SOT223. I'll put a scope on the supply as suggested but don't expect to see problems there. The external hardware on the board consists of an SGTL5000, a PT8211, and the ILI1394 touch display with its led pin driven form a Teensy i/o pin.

When you say the RT1062 starts with a clean slate, that implies nothing in my FLASH code / data / usb / dma or interrupt setup should be able to affect the subsequent loader operation. So I assume trying to put the Teensy in a dedicated "firmware update" mode where it does essentially nothing before updating the firmware won't help?

p.s. I removed a comment about the macos version of the experimental teensy loader with extended time-out in view of the potentially confounding issues I'm seeing. But it didn't fix the problem on the one test system we tried it on, unlike the windows build which has been tested on a couple of different systems and works just fine every time.
 
Last edited:
rvh said:
When you say the RT1062 starts with a clean slate, that implies nothing in my FLASH code / data / dma or interrupt setup should be able to affect the subsequent loader operation. So I assume trying to put the Teensy in a dedicated "firmware update" mode where it does essentially nothing before updating the firmware won't help?
Click to expand...

Correct. Nothing your code does to the state on the RT1062 should have any impact. The bootloader uses hardware reset of the RT1062 so even terrible programs can't put the RT1062 into a state where the bootloader is unable wipe away all effects and fully gain control to allow new code loading successfully.

However, flash erase is slow. It's done in 64K blocks. So the number of blocks needing erase does impact the time taken to load new code. It's not a matter of anything your code does while it runs, just simply its overall size in terms of the number of 64K blocks which need to be erased.
 
Thanks. For the macos build, I was told the Teensy loader times were noticeably slower than for the windows version. Is it possible to extend the macos version timeout even further for test purposes, e.g. to 2 minutes?
 
PaulStoffregen said:
Correct. Nothing your code does to the state on the RT1062 should have any impact. The bootloader uses hardware reset of the RT1062 so even terrible programs can't put the RT1062 into a state where the bootloader is unable wipe away all effects and fully gain control to allow new code loading successfully.

However, flash erase is slow. It's done in 64K blocks. So the number of blocks needing erase does impact the time taken to load new code. It's not a matter of anything your code does while it runs, just simply its overall size in terms of the number of 64K blocks which need to be erased.
Click to expand...
It's interesting that without the display the erase time seems (probably just) short enough for the unextended version of teensy loader to work reliably. Whereas with the display connected I get a download error the first time, and success every second time because (I assume ) the erase is already done. If it was a power supply problem, wouldn't you expect more erratic behavior? I looked at the supplies btw, and there is quite a bit more noise on the regulated 5V supply to the Teensy when the display is present as expected, perhaps up to 200mV p-p, but that seems unlikley to be enough to upset the bootloading process.

(added later) Adding a 220uF cap right on the T4.1 +5V pin to the ground pin next to it makes no difference (the regulator driving this already has 47uF + 0.1uF on input and output pins).
 
Last edited:
Before I make another custom build, can you tell me specifically which Teensy model you're using? Or if a custom PCB, which flash chip it has?
 
PaulStoffregen said:
Before I make another custom build, can you tell me specifically which Teensy model you're using? Or if a custom PCB, which flash chip it has?
Click to expand...
Teensy 4.1, lockable (I'm just about to test to see if I get any different behaviors for hex and ehex files)
(with no ethernet in case it makes a difference)
 
Last edited:
As it turns out the display presence/absence was a red herring. Instead, as I anecdotally noted in the other thread where I wrote about this issue, there is variation across my T4.1 boards where some have the timeout error on the erase cycle and others don't. This is when they're plugged into the same custom host board that provides +5V and peripherals. The T4.1 variation is consistent across host boards (with or without displays), and also appears unrelated to whether hex or ehex files are used.

The variation appears to be well correlated with how often the T4.1 has been programmed previously. I tested a dozen brand new boards, 5 via usb power and no peripherals, and the rest powered by my custom host board with a display present. None of those showed the timeout error with the unmodified teensy loader. I further tested 6 boards that have been used with previous variations of my code to various extents, and a few of them extensively so. Of those, 4 boards showed the erase timeout error, and I'm pretty sure 3 of them have seen quite a lot of development use. One of the 4, however, I thought had been used only very sparingly. If so perhaps it's more the flash space that's been previously used, rather than how often it's been modified, that correlates with the error.

Hopefully this will give Paul or others some insight as to what may be going on with this issue.
 
PaulStoffregen said:
Before I make another custom build, can you tell me specifically which Teensy model you're using? Or if a custom PCB, which flash chip it has?
Click to expand...
Hi Paul, just wondering if you've had any more thoughts about a way to solve this issue. I did some searching and it appears flash erase times do increase over time as the number of previous erase cycles increases, which correlates with my observation that new T4.1 boards weren't failing the teensy-loader timeout with the very large files, but ones that had been programmed more often were.
I've had to give up on allowing end users update the firmware of my T4.1 based device for now, but would really like to provide them with that option as soon as there's a solution that allows it reliably.
Thanks, Richard
 
Can you provide the output of Arduino that shows the code and data sizes? Do you use the emulated EEPROM? Only Paul can speak to the details of the bootloader, but I've done a lot of experimentation with erasing and writing the main program flash from within a sketch, and I've never noticed a change in the erase times with age. I was never specifically looking for that, so it's possible it is there, but I feel like I would have noticed if there was a significant change.
 
joepasquariello said:
Can you provide the output of Arduino that shows the code and data sizes? Do you use the emulated EEPROM? Only Paul can speak to the details of the bootloader, but I've done a lot of experimentation with erasing and writing the main program flash from within a sketch, and I've never noticed a change in the erase times with age. I was never specifically looking for that, so it's possible it is there, but I feel like I would have noticed if there was a significant change.
Click to expand...
Thanks for the reply. Yes, I am using the 4k eeprom. Below is the output for my current build. The high FLASH and RAM2 use are due to including as many flash-embedded audio samples and the longest stereo delay line I can fit into the Teensy. Paul's extended timeout experimental teensy loader builds so far have worked on a couple of Windows machines, but have remained problematic for two test users on MAC OS, although they were using older versions of the OS.

Memory Usage on Teensy 4.1:
FLASH: code:385280, data:7289808, headers:9004 free for files:442372
RAM1: variables:106656, code:381704, padding:11512 free for local variables:24416
RAM2: variables:513216 free for malloc/new:11072
 
You must log in or register to reply here.
Back
Top