Are you using 1.48 or the latest beta?

https://forum.pjrc.com/threads/58817-Teensyduino-1-49-Beta-4

Some bugs with I2S2 were recently fixed, so best to use 1.49-beta4 if you're using I2S2.