It depends on what your external power is:
If your external power is exactly 3.3v, you can plug in the power to one of the 4 3.3v connections (one on each side, one in a through hole in the middle, and one on a solder pad underneath the Teensy). You would plug the ground wire into one of the 6 ground connections (one on each side, one through hole in the middle, one solder pad, and two through holes on the inside that is the USB host on the 3.6 and two additional analog inputs on the 3.5). You do not use the analog ground that is physically between the VIN and main 3.3v pins.
If your power is 3.5-5.5v, you want to plug the power into the VIN pin and not the 3.3v pin.
Alternatively, you could fashion a USB micro-B cable, and use that to power the Teensy.
If you are providing power, you probably want to cut the trace between VIN and VUSB, or you want to unplug the external power when using the USB. This disables the USB plug from providing power to VIN.
As to why there are multiple 3.3v/ground connections, ultimately it was Paul's choice. If you are doing breadboards, having each ground/3.3v on each side, allows you to simplify wiring the wiring to the power rails (i.e. you don't have to have one cable to join the 3.3v and ground between the right and left sides). Similarly for the pads underneath the Teensy, it is useful to have power/ground there, so you can just attach things directly to the pad. For the pins in the middle (reset, program, ground, 3.3v, RTC power), it is useful to have a ground pin there.