While not good, normal I2C pullup resistors to 5V should not kill Teensy 4.0 or 4.1. The chip has ESD protection diodes. NXP doesn't rate them for handling any level of continuous current, but in reality they can pretty easily handle a couple mA.
As a quick test, I put two 1K pullups to 5V on this Teensy 4.1.
View attachment 20404
I ran the Wire library Scanner example, and also the LED blink example (to test what happens if the pins aren't in I2C mode). My voltmeter says both pins are at 3.9V. I can confirm this Teensy 4.1 is still working.
I can't say what happened to your Teensy, but hopefully this helps to rule out the I2C pullup resistors.