Kurt's extra pinout diagram is the easiest way to see which pins can be used with FlexIO.
https://forum.pjrc.com/index.php?threads/teensy-4-1-where-do-i-connect-to-use-third-spi.66144/#post-268934
Looks like 24 more pins not conflicting with...
The one gotcha with FlexIO is the very fast default clock speed isn't good for slow baud rates (almost anything under 115200).
See this MIDI example for how to best configure FlexIO so you get accurate MIDI baud rate.
For expansion, I'd recommend using the USB host port, normally used with this cable. If you connect a powered USB hub, you should be able to plug in at least 4 more Teensy 4.0 or 4.1, and theoretically perhaps many more.
Teensy supports USB MIDI...
I would like to build a MIDI processor for a large midi setup.
it would have 16 MIDI ins and 16 MIDI outs and ideally have a way of chaining units together to make it even larger.
what would be the best approach to getting so many serial ports...