I am currently building something similar, so maybe you can save me some experimentation time. How are you doing the Teensy to Teensy communication. My last multi processor design was in the 1990's using MC68HC11 chips and I used SPI. Here SPI, I2C, and standard serial ports are all viable options. Have you, or anybody else out there tried to pass audio from one Teensy to another in the digital domain?
When I discovered the Teensy and the audio library several years ago the first thing I did was to make a simple two "oscillator" 1V/oct synth with a T3.2. A year or so later I made "Blue" a four oscillator 1V/oct synth with another T3.2, where I found the limits of the Teensy. I got some T3.5's and T3.6's from the Kickstarter build, and made a new T3.6 synth engine for Blue. It took some tweaking but it can "play nice" with my Behringer Model D and Deepmind. The T3.2, 3.5 and 3.6 all have good enough A/D converters for accurate pitch from a keyboard gate / CV out. The T4.X does not. The A/D can be used for pitch bend, modulation, or envelope tracking, but not absolute pitch.
When the Teensy 4.1 came out PJRC sent me one, so it was time for another synth build. This one was small, portable and battery operated, played by MIDI only, no gate / CV. It's mission was to find out how many concurrent synths I can run in a T4.1. As you stated having the T4 read 10 encoders and a touch screen, update the screen, and make music all at the same time can be a limiting factor, so I am making something new.
Blue separated the UI (49 pots and 4 encoders) from the synth engine, so that the synth engine can be swapped out easily, as it plugs into the UI board. I am doing the same on the new synth. I'm building the UI on perfboard now. It has 16 slide pots, a 3.2 inch touch screen, several encoders and rotary pots, and lots of blinky light stuff. I have not yet decided whether to put it's T3.6 on the synth engine, or on the panel itself, there are advantages to each approach. The synth engine will be physically large enough for multiple Teensys, a Tsunami WAV player, a Daisy Seed, an FX board or two, and a discreet built Moog ladder filter (because nothing else sounds just like it).
View attachment 24935 View attachment 24936