Flightsim USB Stack, Teensy freezing

[jbliesener]

I'm stuck with a problem that I believe to be in the USB flightsim stack. The attached minimal sketch demonstrates the issue. It doesn't require any additional hardware and only needs X-Plane with the plugin running when the Teensy starts up. No inputs or external wirings are required. The sketch should blink the Teensy's LED in regular intervals after an initial startup phase, instead, it freezes on the first call to FlightSim.update().

After observing the problem on Arduino 1.6.5/Teensyduino 1.27(?), I updated to Arduino 1.6.9/Teensyduino 1.29 beta. No change.

To me, it looks like a very subtle timing issue when sending and reallocating USB buffers from the low-level Flightsim communications (FlightSimClass::xmit and related calls to the USB stack). Maybe, that could affect other USB devices as well if they implement a similar buffer handling.

The freeze occurs as soon as you have more than 16 outstanding packets on the first call of FlightSim.update(), as soon as the identify-functions are called for each variable.

All of those packets are below the threshold size that would trigger my "xmit_big_packet" function, that I contributed a while ago. Also, those packets ARE actually sent to X-Plane (I can see it on the Plugin's comm-log), but the buffer doesn't seem to become available to usb_malloc to be reallocated.

I suspect a timing issue, as uncommenting the serial_print-functions in usb_malloc makes the program behave differently: only a single packet is transmitted to X-Plane. If I uncomment the serial_print-functions in usb_free, the sketch seems to run. There are other ways to make it run:

• Reduce the number of datarefs to 16 (see comment in sketch)
• Increase NUM_USB_BUFFERS in usb_desc.h to 32

None of them work for our practical case though, as we with about 42 datarefs in our sketch and NUM_USB_BUFFERS is limited to 32, due to the 32 bit usb_buffers_available bitmap.


I also tried playing with the CPU frequency and optimization settings, and at one point the program was actually running when I set it to 24 MHz. I sent the sketch to raflyer, but he couldn't reproduce it. After upgrading to 1.6.9/1.29beta, I wasn't able to reproduce it either.

As those USB stack functions are very deeply integrated in your USB stack, I don't feel confident enough to make any changes. Maybe, it's only about moving a __disable_irq() a line up or an __enable_irq() a line down, but I'm unsure about the implications.

Could you take a look at usb_flightsim.cpp, maybe especially at FlightSimClass::xmit and tell me if you see something wrong?

 

[jbliesener]

Just to add versions:

X-Plane 10.45 (doesn't seem to matter)
X-Plane-plugin from Github
Arduino 1.6.9 (same problem with 1.6.5)
Teensyduino 1.29beta (same problem with 1.27)

Teensy 3.1 or LC, 96, 48 or 24 MHz (optimized or not), USB Flightsim

Problem occurs with default Cessna sitting on runway threshold in X-Plane. No external hardware required, no other inputs required, nothing special to observe on X-Plane. The problem is related to USB communication between the Teensy and X-Plane and it looks as if the root cause is on the Teensy side.

 

[jbliesener]

New discoveries - gets even weirder...

Tracking it down further. raflyer gave an important clue. It also seems to have to do with the initial delay(5000) statement from line 7 in the sketch. Reducing this delay() brings the sketch alive.

The attached modified sketch runs on a Teensy 3.1 (96 MHz, optimized)and prints the value of millis() at the beginning and end of startup() - and therefore, right before loop() is called. Also, it now sends debug output to HardwareSerial1, in order to not interfere with USB I/O.

From what I'm seeing right now, the sketch seems to run if the first FlightSim.update() occurs within about 2000 ms after program startup. However, the exact value varies. I've seen it running with 2170 and, in another instance, it doesn't run with the same value. However, below 2000 it always seems to run and above 2200 it hardly ever runs.

I know that makes it even more weird to debug, but, at least I have a workaround for my problem now.

---

Various tries. 2199 sometimes works, mostly doesn't. 1999 always works:

Startup, millis: 1999
Setup done, starting loop, millis: 1999...........
Startup, millis: 1999
Setup done, starting loop, millis: 1999.....
Startup, millis: 1999
Setup done, starting loop, millis: 1999.....
Startup, millis: 1999
Setup done, starting loop, millis: 1999............................................................
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199.........
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199
Startup, millis: 2199
Setup done, starting loop, millis: 2199