Maybe your speaker was playing the signal of your screenshot. The signal is just very quite. Anyway, there is a problem. Can you remove the audioshield for now and just concentrate on receiving the correct signal from the ESP32?

Ok, I would like to summerize what I found out:
I had a look at three different possibilities to connect the ESP32 to the T4. At all three options I configured the I2S signal for 16bit samples. I wasn't able to get it working with 32bit samples.

1. The T4 is I2S slave and the ESP32 is master:
Example code can be found here:
This seems to work, but I think there are then many restrictions on what can be done with the audio signal at the Teensy. Using for example the spdif output would probably not work well since it is clocked by the PLL4 of the Teensy.

2. The T4 is I2S master and the ESP32 is slave:
Example code can be found here:
I think I found the correct I2S configuration of the Teensy, but zeros were inserted into the audio signal. I found a thread were people were discussung some problems when the ESP32 is running as I2S slave:
It seems that the ESP32 can only work correctly in slave mode, if its internal master clock is in sync with the provided bit and frame clocks. This was obviously not the case at my experiments since the clock at the Teensy is generated by its internal PLL. However, the T4 offers lots of possibilities at the clock routing and it is maybe possible to use the master clock from the ESP32 to clock the I2S interface of Teensy. Then this configuration should work.

3. The T4 runs in slave mode, but the input is resampled:
The files from are needed and the example is here:
At this example the audio data is resampled and forwarded to the spdif output. The plots of the wave looked correct and the signal sounded fine over my active speaker. The example works with the first I2S input.
If needed, I would provide an example for the second I2S input/ SAI2. Then the first I2S interface can be used for the audioshield.