The big leap would be a chip with support for USB3 superspeed. Teensy has always been a leader in USB port throughput for Arduino. Also DDR4--leaping up to 1 GB would sure feel nice after so many years trying to shoehorn our code into 2 MB.
Also, while the clock speed is about the same as Teensy 4.0, there are more cores, e.g. the AM2434 is a quad-core (and $13 on digikey), so the raw computing power is 4X. This is the sort of logical leap that all computing has taken over the years, moving from single-core to multi-core to get more computing power, so it feels logical that the premier embedded-performance platform would eventually go that way.
I *think* (?) that these chips have general purpose high-speed serial/parallel interfaces customizable through registers that could likely implement I2S. (and I believe it still has 3.3V pin interfaces) Also, the cores can run partitioned from each other, so for example one core could input and output a 24-bit 192khz audio stream without glitches, while the other cores could implement digital effects or mixing etc., i.e. without worrying about time-slice threading all that on one core without audio drop-outs. On the "con" side, the technical manual for AM243x has broken the 10,000 page barrier... oof.
Ooh, one thought I just had is that maybe one of the cores could run RTOS, while the other cores run bare-metal Teensyduino, then, Teensyduino could use *all* the pre-coded peripherals available under RTOS--ethernet, PCIe, USB3, etc, etc. and just use inter-core communication to let Teensyduino request such services from the RTOS core. That would save all the massive effort to custom-develop and debug/test such services for bare-metal, which I imagine is a big part of the man-hours in releasing a new Teensy generation?
Obviously, adapting Arduino to take advantage of multi-core is a whole can of worms... but an *interesting* can of worms, haha, that I could imagine igniting a curiosity in a particularly inventive mind to attempt to harness.