Pots on a synth are great until the ADC noise creeps in, fwiw — a simple exponential filter on each analogRead cleans it right up. Wrote more about the Teensy audio setup and keeping controls stable here...
Try slapping `volatile` on that asm block. If the compiler's optimizing differently now, that'll keep it from messing with your timing. Could be the issue.