All 3 speeds, 24, 48 and 96 MHz, are created from the same 16 MHz crystal using a phase locked loop, so they all have essentially the same accuracy. The crystal we use on Teensy is a good one, rated at +/- 15 ppm initial accuracy (at 25C) and +/- 15 ppm over temperature (-10C to +75C).
Freescale specifies the chip at 50 MHz maximum, so using the 24 or 48 MHz speed is the most conservative.
Freescale makes several other similar chips, which are rated at 100 and 120 MHz. They are all fabricated on the same 90 nm silicon process. They all use the same ARM core (but some add extra options) and basically the same on-chip peripherals. Normally I would be cautious of overclocking. However, in this case we know the same ARM core runs at 100 MHz and higher on other chips made with the exact same silicon process. I believe the risk of damage, or even improper operation, is very low.
However, 48 vs 96 MHz is not always as simple as it might seem. Only the CPU core, DMA controller, crossbar switch and a couple other things actually run at 96 MHz. There several clocks inside the chip. The timers and most other peripherals run at 48 MHz, even when the core is at 96. The flash memory is relatively slow, with a small cache and additional buffering inside the ARM core. Most instructions are 16 bits, but some are 32 bits, so the memory bandwidth requirements vary in complex ways depending on the code. At higher CPU speeds, more cache misses occur when running from flash, so the actual execution speed does not always increase linearly with the CPU clock.
However, the timers based on the PLL derived clocks are very accurate, because the clocks all come from a high quality crystal.
However, in terms of controlling LEDs on the time scale of a song, these speeds are incredibly fast relative to human perception of light and sound. Unless you're doing something incredibly complex, these sorts of timing probably aren't a big deal?
any chance you can share the exact crystal that is in the teensy 3?