Audio input on the ADC pins is still a work in progress on Teensy 4. So far only mono exists, and it's best described as "experimental" at this point. It will eventually improve, but perhaps not until sometime in 2021 to be realistic.
On Teensy 3, you can't make use of the ADC with analogRead or the ADC library while the audio library is using it.
Teensy 4 has a peripheral called ADC_ETC which is meant to allow multiple threads or unsynchronized programs to share the ADC. I'm happy to say the (still very experimenal) audio library code is making use of ADC_ETC. But so far analogRead() is not. I also hope to eventually update analogRead() to access the ADC through the ADC_ETC rather than directly using the ADC's registers. In theory that could allow use of analogRead() on the same ADC hardware as the audio library is using. It might also allow for very reliable use of analogRead() from both interrupts and main program context.
So to specifically answer your "Is there a way to do this" question, on Teensy 4 we do have this pretty amazing ADC_ETC hardware which (probably) does provide a way to reliably have both audio and main program share access to the same ADC hardware. But the software support to easily make use of that hardware is still in its infancy.
If you want to explore the ADC_ETC hardware, it's documented in chapter 67 of the
IMXRT1060 reference manual (rev 2) starting on page 3371.
The Teensy 4 audio library code using it can be found in
input_adc.c starting on line 216. The basic idea is one of the timers sends a sample rate trigger pulse to the ADC_ETC, which is programmed to access the ADC and send trigger events to the DMA controller. One of the other fairly experimental aspects of the code is capturing (oversampling) at 4X the audio sample rate and a low-pass filter. Removal of DC offset, details of that filtering, dealing with slight mismatch between the timer and audio sample rate, and setting the overall audio level after all this processing are still things in need of much more work and testing.
Eventually I will do more work on this stuff, but that is unlikely until well into 2021. I'm currently working on a variety of things involving the bootloader and new hardware. I'm just 1 guy and I can't do everything all at once. I need to prioritize my dev time. Sadly, that means this ADC stuff isn't going to get a lot of work until well into next year.
But for some good news, the answer to "does the 4.1 support stereo while doing" is good. Quad I2S works, and it's fully independent of this ADC stuff, so it can indeed be used while doing the currently-limited ADC input.