That method will work, but will not be particularly efficient, wasting around half the energy in the battery.
If that is Ok then go for it and have fun, otherwise you start trading cost and complexity with switching regulators, that will be more efficient but complicate things.
A half way step is things like this https://www.adafruit.com/products/1065 (not just from adafruit) which can be dropped into the above circuit but cost more.
Another option are the USB battery banks, though a Teensy can turn out draw so little they auto shutdown.
If you can go with a rechargeable USB battery pack I've found a brand that is working well for me posted in this thread. It has no low power shutdown and can be charged while providing current.
The end of the thread lists a switching buck converter from onehorse that mates to a Teensy and takes higher DC voltages to 5v.