I wonder if anyone has any experience with this problem before...

I recently noticed that my most recent Teensy 3.2 based project wasn't executing code occasionally, maybe 1 in every 20 power ups. I did some investigation and this is what I've found so far:

1. When the Teensy fails to execute code, 3.3V is still present and the teensy is drawing 20mA from my +5V supply compared to 40mA when the Teensy is executing code.
2. It seems more likely to happen during quick power on-off-on cycles, but also happens during a power on after a long duration of being off.
3. One of the things I tried was different value capacitors on my +5V supply and Teensy's 3.3V supply. All combinations, including no additional capacitors would still occasionally have the Teensy not executing code on power-up.
4. If a large value capacitor was placed on Teensy's 3.3V supply and left to slowly discharge after power off, powering on my +5V supply just as the +3.3V was at 350mV would almost always guarantee that the Teensy would not execute code!

I'm at a loss for this one, but #4 is pretty interesting.. so I can't help thinking there's some sort of weird condition happening during power-up that hangs the Teensy. Also of note:

5. The behavior is the same with the Teensy 3.1 that I also have, so it's not an issue with the Teensy itself.
6. It is also not code related, I uploaded the blink sketch for my tests.

About my setup:

I am powering the Teensy with +5.00V applied to the Vin pin with the VUSB trace cut on the back (but no USB cable present for the tests). My +5.00V is coming from an L78L05 with a .1uF ceramic capacitor on the output and nothing else connected to it.