I have the same issue with a teensy 3.2 on OSX
I think it is more an issue of the USB, because after you turn off your sound for a couple of seconds and turn it on again it's back to normal. Out of my head i guess it has something todo with the buffer of the USB connection.
I tested it with:
-...