Look at the bottom side of both circuit boards. You should see pin numbers labeled on both. You need to orient the 2 boards so the numbers all match up.
Also quickly check that the audio shield says "Rev D". The older versions have wiring for Teensy 3.x boards.
It doesn't matter if Teensy is located above or below the audio board. As long as the pin numbers match up, the electrons don't care. While it also doesn't matter if you use sockets, soldering sockets to 1 board and pins to the other is really convenient if you ever need to separate them for troubleshooting or use in another project.
If you hard solder them together with just pins, make sure you leave some clearance between the 2 boards. We've had people ruin their boards because they were too close, especially when if the parts on the bottom side of Teensy 4.x touch the top of the SD socket on the audio board. Simple as this sounds, double check you have enough clearance so they're definitely not touching before soldering.