If you want something that is made to be soldered to a Teensy consider this. You will have to cut the solder connection underneath the Teensy that connects VIN and VUSB:
Alternatively if you what to use a second USB port for charging the battery consider these products:
The adafruit board hooks up a 3.7 lipo battery and provides JST output that can be hooked to VIN/GROUND. The Seeed product takes the battery and produces 5v that can be plugged into the Teensy's USB port. Both of these can charge batteries faster. But you likely want to use batteries with at least 1,200mAh capacity.
<edit>
And the thread that kd5rxt-mark posted also contains my recommendation for the Adafruit featherwing adapter, which works best for Teensy 4.0 due to the size (Teensy 4.1 needs to be mounted taller or underneath the board). If you want to use the featherwing adapter for featherwings, there are some addition caveats I can point you to.