Depending on the final goals for this project (are you planning to sell it? as a kit or a 'finished device'?) you might be fine and it might be terrible.
USB can be finicky - if you do use jumper wires from the pads, make sure they're all *Exactly* (or very, very close) to the same length. USB is a differential (I think that's the right word) serial connection - it sends the signal on one trace, and the inverse of that signal on the other, for noise reduction reasons. If the two signal traces are different lengths, you can get mismatched signals and nothing will work (the positive and negative signals get out of sync with each other).
Every USB application note and design doc I've seen (which, granted, is not that many) has said a 4-layer board is a minimum requirement, with the layer closest to the USB traces being a ground plane. Again, those neg/pos signals can be affected by a lot of weird stuff, and anything that knocks one of those traces around (strange board capacitance, crosstalk, etc) can render the USB signal useless. Also, make sure you USB signal traces on the board are the exact same length, follow the same path, and are the same distance from each other at all times - avoid vias if at all possible (USB signal traces should be as straight a shot as you can make them).
If you're planning to sell this as a finished product, and you're in the US/Canada/UK, you'll most likely need to do an FCC declaration of conformity - which involves some very specific tests with a radio antenna and a very nice oscilloscope (IE, you'll need to take it to an RF test lab, unless you happen to have about $50k of test equipment laying around). For the UK (CE cert) you'll also need to do immunity testing, which is where they fire radio noise and frequencies at your device to see if it breaks. I mention all this because USB is noisy as hell, and you'll likely see loud peaks at 12, 24, 48, 96, 480, and 960MHz (and probably more in between), if you're not VERY careful about the EMC design of your board. I had to use a common mode choke on the data signals to clean out some USB noise from my device - which is already a 6 layer board with good ground planes, decoupling, and a grounded chassis acting as a faraday cage. USB is noisy as hell.
Keep the USB traces far, far away from any crystals or anything else that would have a 'loud' oscillating frequency - ESPECIALLY a frequency near the USB rates (12Mhz and multiples thereof). Avoid anything running parallel to the USB lines on other layers. Keep the USB traces as short as possible.
If I think of anything else I'll pop my head back in. Honestly if it's just a one-off thing, the simplest solution is to find a USB cable that you can plug in, because they're (hopefully) shielded and all the wires are the same length. If you're planning to make a lot of them, and have a bunch of other SMD components that'll need to be machine placed anyway, it can be better to just build the teensy onto the board itself as a custom design, buying the bootloader chips from Paul.
