Changing the pitch of a sample is complex since it comprises extrapolating, filtering and re-sampling which is most probably too much work and mathematics for a small embedded MCU like the Teensy 3.2. I'd rather think of generating up to 127 samples (one for each possible midi pitch within the pitch range you are interested in, but for drums, it doesn't probably make sense to go above midi note 84) and to put them on the SD card as 60.WAV, 61.WAV, and so on. Then, you can then trigger the playback of these samples with whatever you want. Or, still simpler, use the drum synth object from the audio library.
In ever case, the decay can be easily handled by the envelope object of the audio library.