Pretty much all the stuff the audio shield can do (for example, in this
45 minute video) is because of Teensy 3.x and the Teensy Audio Library.
The audio shield pretty much just gives you a stereo ADC and stereo DAC. It does have an impressive list of features for adjusting those, like programmable analog signal levels. But all the features like playing 2 files, software panning, mixing, oscillators, effects, etc all come from the Teensy Audio Library.
If you connect to something else, the actual capabilities will depend on what software exists on that other board. It's easy to take software for granted. It's easy to think only of the DSP extensions without understanding the tremendous amount of work that goes into effectively using such hardware. I can tell you it's taken years of work to build the audio lib. Doing things well, especially to allow doing many operations and processing many sounds at the same time, while also being compatible with most other Arduino libs, it no small task!
Years ago there was a "codec shield" for ordinary Arduino boards. As far as I know, nobody ever got playing of WAV files from a SD card working, but they did manage to generate some waveforms and create some simple effects... after quite a lot of work!
If you do try to connect to something else, two important details to consider are MCLK and I2S master vs slave mode. The SGTL5000 chip requires MCLK. It won't even come out of reset mode and allow config by SDA & SCL until you give it a clock on MCLK. Some microcontrollers have only BCLK and LRCLK on their I2S ports, but not MCLK, so if you try to use with one of those you'll need to do *something* to get MCLK. The audio lib uses I2S master mode (Teensy generates BCLK & LRCLK), so it always configures the SGTL5000 chip in slave mode. Some microcontrollers with I2S don't really support master mode.