Teensy 3.6 Serial after unplugging USB

DavidBell

Member
Hello

I have a T3.6 board using Serial (ie USB) that works fine immediately after it is programmed. But if I unplug/replug it at the USB then it misbehaves - sometimes extra characters, sometime freezes always consistent in the misbehaviour.

I can fix it by programming again, or using Reset in TyCommander (which I think activates Hafkey reset but doesn't reprogram it), both get it working properly again. But as soon as I unplug/replug again it's back to misbehaving.

Is there anywhere I should look for the problem? It is as though it is not initialising properly when just plugged in, possibly even at the USB end.

(Teensy 3.6, Teensyduino 1.41, Arduino 1.89)
 
That version of TeensyDuino is way behind 1.56 release or 1.57 in beta. Updating and confirming and posting a repro test sketch would allow others to perhaps see.

TyCommander is not officially supported - but tend to use it exclusively here due to high awesome factor. Generally TyComm is quick and effective at picking up a restarting Teensy without issue. Using it so much from T_3.6 Beta until moved more to T_4.x's this has never shown as an issue here.

If possible and updated TD 1.56 shows the same issue, confirm the behavior with the IDE Teensy Serial Monitor.

TyComm triggering 'Reset' does IIRC tend to trigger TeensyLoader when active to program. There is no simple USB 'Reset' coded in a Teensy, so the Teensy Loader will try to jump in.
 
That version of TeensyDuino is way behind 1.56 release or 1.57 in beta. Updating and confirming and posting a repro test sketch would allow others to perhaps see.

TyCommander is not officially supported - but tend to use it exclusively here due to high awesome factor. Generally TyComm is quick and effective at picking up a restarting Teensy without issue. Using it so much from T_3.6 Beta until moved more to T_4.x's this has never shown as an issue here.

If possible and updated TD 1.56 shows the same issue, confirm the behavior with the IDE Teensy Serial Monitor.

TyComm triggering 'Reset' does IIRC tend to trigger TeensyLoader when active to program. There is no simple USB 'Reset' coded in a Teensy, so the Teensy Loader will try to jump in.

I'll update and try again and also try and put together a code snippet, but I've found it difficult to track down a common factor. It does seem to happen in a std::vector (that is brace-initialized, so in principle doesn't use new()).

Worst case, this is all connected to a PC which can issue resets via TyCmd but I'd like to understand what's happening.

What is IDE Teensy Serial Monitor? I have been looking at it with my own monitor and the TyCommander monitor.
 
In the IDE with a Teensy online the 'Teensy Ports' will show the Teensy devices. Using that PORT will use the PJRC customized for higher speed Serial Monitor code.
TeensyPorts.png
 
.... works fine immediately after it is programmed. But if I unplug/replug it at the USB then it misbehaves ....

Quick blind guess, try adding a delay at the beginning of setup().

Often when other people have reported problems where a program works after uploading, but fails when power cycling the hardware, the problem has been Teensy boots up too quickly and some other hardware isn't ready or fully working when setup() runs... but is ready when you upload because it was already powered up during the upload time.

Of course, I have no idea what other hardware you're using (if any) or what your code looks like. This is just a completely blind guess, because your description of the problem sounds similar to so many others which turned out to be Teensy booted up too quickly.


(Teensy 3.6, Teensyduino 1.41, Arduino 1.89)

Latest stable version of Teensyduino is 1.56. And 1.57 beta testing is started.

If you really are using very old 1.41, maybe try installing 1.56 or 1.57-beta1. Probably will not change anything, but installing the new version is very easy to try.
 
Thanks for your help. I updated the toolchain and added 5 second delays but it's still not right. PS. Doing it with the bare board now too.

I've been trying to isolate the problem code by going through older versions, but it turns out it goes a long way back (I must have been developing with it always plugged in).

One thing, I have been using an updated compiler from the g++ normally supplied (updated to gcc 11.2 from 5.4) because it uses some c++17. All the libraries seem to compile OK, but would that cause a problem?

I've also attached a picture of the problem. It's probably not very helpful, but it's worth noting that the within a run the error characters are always the same, but power cycling changes them. Which makes me think there is something not initializing correctly.

Teensy Serial Error.jpg
 
That looks like output that would be expected when the '- Fail ...' output string isn't 'Null' terminated.

The 'garbage' looks consistent on the two runs - and the first couple of garbage chars are the same on both runs.

Not seeing any code presenting the output can't guess or rule out memory corruption.
 
One thing, I have been using an updated compiler from the g++ normally supplied (updated to gcc 11.2 from 5.4) because it uses some c++17. All the libraries seem to compile OK, but would that cause a problem?

Compiler bugs are pretty rare, but several times we've seen use of a different compiler trigger subtle bugs like timing issues or missing use of volatile.

For one example, here's a fix which went into version 1.54.

https://github.com/PaulStoffregen/cores/commit/ee3a0ada4af8338bc24f1cf12eb696dc6aa02b3b

So if you're using an old version of the core library code with a newer compiler, you're probably going to hit this bug. The old workaround to change the compiler optimization level probably only works with gcc 5.4. This change committed about 1 year is the proper fix which should work with all compiler versions.

It's theoretically possible more subtle issues like this could be lurking and (so far, at least to me) unknown. Almost all testing is done with the gcc version we ship with Teensyduino. It hasn't been updated for a long time because most newer versions has performance regression - compile code simply didn't run as fast.


Which makes me think there is something not initializing correctly.

Could be. Could be any number of other complicated problems. These sorts of things aren't easy to diagnose even with a reproducible test case. But with blind guesswork over the internet, pretty much impossible to say.

If you don't find a solution, maybe try creating a test case, ideally which reproduces the problem using Arduino 1.5.19 and the latest Teensyduino version, which today is 1.57-beta1.
 
Thanks Defragster, I'm 99% sure it's not a null terminated string problem. It seems to consistently fail in my std::vector<my_struct> so I'll look into the initialisation there.

Paul, I have the more recent usb_dev.c, but I checked a few optimisation levels and it does change the error a bit though no fix. Oddly -O0 is worst, -O3 best.

When I get it sorted I'll post back.
 
It does seems to be something with initialisation. When I look at it's memory address of the vector object (*) after reprogram/reset it is always 2002FF74 (537,067,380), whereas after power cycle it moves around 45EF3000 (1 173 303 296) which I don't think is even a valid address for T36.

The vector object's containing object always turns up at 20017670 (536,966,768) regardless of how it is started, but the location of the vector object moves. Also checked, the memory location is wrong immediately after creation, before any code has run.

(* = The vector object is a class that contains a std::vector)

PS. Tomorrow I'm going to swap the std::vectors for std::arrays, so move all memory allocation to the stack. They are not resized at any time but the vectors were a bit more convenient than arrays (there are several of them of different sizes which with array means multiple versions of the class, but I have enough memory to just make them all std::array of max size)
 
Last edited:
So I have this about 80% solved, it's pretty obscure.

It's something to do with some class level reference variables that I put there just for convenience. For some reason these are not properly initialised on power cycling, but the underlying class structures that they reference are fine. So for eg (may not actually exhibit the problem)

Code:
struct S
{
        int SomethingWithALongName; // always intitialises fine
        int &ShortName {SomethingWithALongName}; // might not initialise correctly on power cycle

        // yeah, I know, not really conventional style, but that's how I roll

};

I still don't understand why the method of start up would make a difference, but that's how it is.

It's still not 100% fixed but I think when all bits of code like this are removed it will come good.

Thanks again for your help.

DB
 
Last edited:
Wild guess: Can it be that your object of S lives in DMAMEM (e.g. new/malloc). This memory is not initialized. Since the ShortName will be assigned the address of SomethingWithALongName at construction time it simply might fail if in DMAMEM

Here some test code:
Code:
#include "Arduino.h"
struct S
{
       int SomethingWithALongName; // always intitialises fine
       int &ShortName {SomethingWithALongName}; // might not initialise correctly on power cycle
        // yeah, I know, not really conventional style, but that's how I roll
};

//DMAMEM S s{42};
S s{42};

void setup()
{
    while(!Serial);

    Serial.printf("%p %p\n", &s.ShortName, &s.SomethingWithALongName);
    Serial.println(s.ShortName);
}

void loop(){
}

This prints here:
Code:
0x1fff0730 0x1fff0730
42

I.e. both ShortName and SomethingWithALongName point to the same memory location which is initialized to 42 as expected. If I place s in DMAMEM I get:
Code:
0x1d1 0x1fff0200
552750155

So, in this case your reference was not initialized (points to some arbitrary memory location). The compiler can't know that the Teensy startup code does not initialize variables in DMAMEM so it won't complain. As mentioned, this is just a wild guess and might be completely wrong ...

Edit:
If you define a constructor for you class (which is cleaner anyway) it works as expected: E.g.:
Code:
struct S
{
       int SomethingWithALongName; // always intitialises fine
       int &ShortName ; // might not initialise correctly on power cycle

       S(int val) : ShortName(SomethingWithALongName)
       {
           SomethingWithALongName = val;
       }
};
In this case the assignment of the variable address to ShortName does not rely on the initialization in the startup code.
 
Last edited:
No, I don't think that's it, yesterday I changed the vector members to arrays so it would all be created on the stack. But it might be related in some way with the order things are initialised.

Meanwhile I'm going have to go back to the books. It thought

Code:
int &ShortName {SomethingWithALongName};

and

Code:
int &ShortName = SomethingWithALongName;

and

Code:
S() : ShortName(SomethingWithALongName) {}

were functionally identical.
 
Interesting. Using the constructor fixed it for the shown example. Can you post a more complete example using your struct showing the effect?
 
To update, I have this all fixed now.

Sort of as luni surmised, it seems there is some memory that get initialised to 0s on Reset but in some power cycle cases does not initialise - perhaps low V early in powerup means the initialisation fails for some segments but not others?

This affected my program in two places (only on power cycling, not resetting)

* One was some class level reference variables weren't correctly. Removed cases, and where I really wanted them replaced with member functions following pattern above
inline int& ShortName {return SomethingWithALongName;}

* A fixed length string class also didn't initialise correctly, leading to garbage up to its terminating null. Added extra initialisation.

Thanks again for you help.
 
Luni,

It's possible that the constructor method would have fixed it, but I had removed the code by the time I saw your message. But I've always assumed those three lines all do the same thing, if anyone knows to contrary I would be interested to hear.

DB
 
Luni, this is more context, very cut down, not sure how much help it will be. Uses CRTP which may seem a bit odd but it's working for me.

DB


moderator note: deleted code
 
Last edited by a moderator:
I am taking down the code above, I don't think it's relevant. I found two other problems.

Well, seems I can't edit the post above. Anyway, ignore it, red herring.

* First was I had a noInterrupts() call before Serial.begin(). I'm not sure it made any difference but I don't know why it was there either so I removed it.

* Second, my computer fan had been whining for a couple of days. I got round to looking in task manager and I found a dead teensy uploader process running in the background. I'm not sure how that got there but I killed the task and everything has been rock solid ever since.
 
I deleted the code from msg #17.

Any chance you could try reproducing the error which caused the upload process to lock up? If there's a way to reproduce it, I'd like to investigate what causes that upload process to stall and prevent it from happening in future versions. But without a program which reproduces the issue, odds of getting to the bottom of this are pretty much zero. Hope you can put that noInterrupts() back in, and if the problem returns, maybe trim the code down to the smallest program which causes the problem.
 
Thanks for the delete. I'll be putting some of the problem code back in the next few days and if I see an error again I'll post here.

But I think the root cause was the extra teensy uploader process, I suspect there were conflicts when uploading with two processes fighting each other. At one stage I thought I had bricked four in one go, windows was reporting them all malfunctioning. When I got rid of the rouge process and reprogrammed it all came good.
 
If it happens again, even just a screenshot of the task manager would help. Details matter.

But code to upload to Teensy and steps to reproduce it on a specific version of Windows would be the best thing. If I can figure out how it's getting stuck, even if it's a problem I can't fix directly, maybe I can add code to detect that stuck condition and cause the program to exit gracefully after a several seconds. Maybe. First I really need to understand the problem better.
 
Back
Top