Upon looking over the Serialx.c code due to a recent issue (posted on this forum), I noticed that at Serial_being, we have RX pin set up as:

The first two flags enable the internal Pull Up on RX which is desirable to prevent noise and false start bits if RX is floating.

However the third flag, PORT_PCR_PFE enables a Passive Filter on the pin. According to the chip manual:
Passive Filter Enable

This bit is read only for pins that do not support a configurable passive input filter.
Passive filter configuration is valid in all digital pin muxing modes.
   0 Passive input filter is disabled on the corresponding pin.
   1 Passive input filter is enabled on the corresponding pin, if the pin is configured as a digital input. A low
pass filter of 10 MHz to 30 MHz bandwidth is enabled on the digital input path. Disable the passive
input filter when high speed interfaces of more than 2 MHz are supported on the pin.
Refer to the underlined text: the Passive Filter is recommended to be disabled above 2Mhz.

AFAIK Teensy 3.2 should support up to 6Mbps; therefore it seems that. At such high speeds, this flag should be disabled (?and may even corrupt data reception?).

I presume newer Teensy's can even go faster which makes this even more of an issue.

I don't have actual code to back this up at this point though, but wanted to post this as a heads up.

Best Regards