Both boards have the pin numbers marked on the bottom side. Maybe sure they are oriented so the numbers match up!
Using pins on one board and sockets on the other results in larger distance between, but it's very convenient if you ever want to disconnect them from each other.
If you solder them together using only pins, and especially if Teensy is on the top, make sure you have some clearance between the SD socket on the audio shield and the parts on the underside of Teensy 4.1. The height of plastic on ordinary header pins is not enough. We've seen several reports on this forum where parts touching between the 2 boards shorted the power and caused Teensy to not work at all.