This should be able to work, if the oscillator is able to drive the on-chip capacitor. A resistor between the oscillator and Teensy pin might be a good idea to limit the current if the oscillator...