This is odd in the sense that I've verified that with USB + Serial, Teensy 4.0 and 4.1 work fine over an eBay el-cheapo ADuM3160-based full speed USB isolator. In fact, that is my typical workflow when I don't need high speed, just to be sure my USB ports won't fry due to my ineptitude. However, I haven't done any MIDI stuff with 4.0 or 4.1 at all. To me, this sounds like very MIDI-specific; is that possible?
I'm waiting for the chipageddon to pass, so that TI
ISOUSB211 becomes available. Mouser has the devkits available (for about 65€). The chip provides low/full/high-speed isolation, with a board complexity similar to ADuM3160, except with more bypass capacitors, added 1.8V LDO, and TVS diodes. With these, a low/full/high-speed USB 2 isolator can be implemented for about $20.
If anyone has any specific tests with Teensy 4.0 or 4.1 and an ADuM3160-based USB isolator (full speed, 12 Mbit/s) in mind, feel free to drop me a private message or email (both are in my profile here); I'd be happy to re-check.