maybe the NCP1117 allows such a setup
2 diodes ? you mean one from battery, one from usb I guess
(...) just use a dual schottky diode like the BAT754C. Connect the 78(L)05 output to pin1, USB power to pin2 and pick the power to the Teensy from pin3.
...
Man, I'm not a native English speaker/writer. But this does look relatively clear, doesn't it?