The SD card on the audio board is slower than the built in SD card on the Teensy 4.1 which is using SDIO.
It is explained in this post:...