Well, I'd try it and see, if you have the hardware (i.e. Segger link, toolchain, etc.) set up. For me, the whole point of Teensy is the ease with which it can be set up and run. Getting a rig working that goes down to bare metal is (IMO) more trouble than it's worth. For a single-chip solution, I'd look at using the DUE chip, that thing is ginormous and has copious I/O, is supported, and so on. More importantly, with the help of VisualStudio and Visual Micro, you can dive into that thing as deep as you want (incl. debugging via SWD with Visual GDB) without having to go super exotic. Atmel ships a SAM version of VisualStudio with all those boards/chip built in.