Confirm, the main reason is to prevent possible back flow of current into your PC. Or the same into whatever power supply is connected to Teensy, if that power supply is turned off while the PC gets connected.
Indeed the risk in practice is...
well that's not the problem, something is shorted and pulling the 5v down to about 2.5v and there is no 3.3v. I'm just going to order a new replacement.