You don't give us much to work with. Is the Teensy being mounted on another PCB where you could use tiny spring contact pins? Could you pick the +5v from the Vin pin on the corner of the Teensy...