How does the compiler know to use the Teensy SD code instead of the generic Arduino code?
The Arduino IDE searches for libraries in many places, which match the #include lines in your sketch. How it matches them has changed somewhat over the many Arduino versions.
In recent Arduino, if you turn on Verbose Info in File > Prefs, Arduino will show you a list of the locations where it actually found the libraries.
If it found more than 1 for each .h file you included, it will also show a message that a conflict was found, with the location of the one it used and other others it ignored. This conflict message is supposed to always appear (but only if a conflict is detected). The non-conflict info only appears in verbose mode.
So how does it know. Well, first Arduino looks in your sketchbook libraries folder, which is usually Documents/Arduino/libraries (but may be configured elsewhere in File > Prefs). If a matching library is found there, it always overrides all others. This is a powerful feature that lets you always be in control, since the library you put there always overrides others. The location is separate from Arduino's folder, so it persists long-term. It's also a real liability if you forget you put a library there, and months or years later it gets used instead of a newer one elsewhere.
Next, Arduino looks in the platform libraries folder. This location is determined by which board you selected in Tools > Boards. Actually, which architecture. All the ones grouped in the same arch share a common location. Traditionally these are in hardware/(platformname)/(archname)/libraries. Recent versions of Arduino also allow these to be installed inside your preferences folder, which varies depending on the operating system. Teensyduino puts a customized copy of SD into its own platform folder, which is always hardware/teensy/avr within Arduino's folder. When a platform lib overrides the global ones, it's not considered a conflict for printing a warning, since platforms need to override global libs.
The global libs folder is the last place Arduino searches. It's the libraries folder you see in Arduino's folder (or inside the app bundle, if using a Mac and you know how to drill down inside it).
it is funny, that I thought this would be easy.
Well, it's supposed to be normally be easy.
As you can infer from those messages and warnings about library locations and conflicts, sometimes it's not so easy. Hopefully those messages can help you understand what Arduino's really doing.