Actually, I'd recommend you do exactly the opposite. Leave the optional memory off until you have a real need. Only a couple very specific objects access the external memory. Everything else makes absolutely no use of either type of memory chip.
This doesn't work like a traditional computer, where extra memory gets used automatically to make things run better.
Even with Teensy's internal RAM, the memory usage isn't like traditional software were allocating more memory allows more stuff to be cached and things run better. The audio library requires a certain amount of memory. It's a hard real-time system. If that memory is available, things work. If not, some or all parts fail. No extra memory is ever used, only the amount needed based on your design.
In fact, the library has
memory usage query functions so you can observe how much of the memory it's actually using. The idea is you can use these to adjust your AudioMemory(number) line at the beginning of your program to only the amount the library actually needs to run whatever audio design you've created. Extra memory just sits there, forever unused.
Again, just to be clear, every object in the library that needs memory for general operation will *not* be able to use the external memory chips. No matter how much memory you add, it won't actually be used by pretty much everything in the library. Only a couple very specific features access that memory. Even those use some of the general purpose internal RAM. They only go to the extra (slower) work to access the external memory for specific features.
Really, your best path is to just start using the audio library without any extra memory. You can use the normal delay object with the internal RAM. You can play sound clips from the SD card or from C files using wav2sketch.
Do those first! If you need a longer delay than you can get from Teensy's internal RAM, then add the 23LC1024 chip. If you want longer clips and the SD card is too slow to layer several of them together, than add the flash chip. By that point, your decision about which chip to use can be guided by the actual sound clips you're trying to use. (and your knowledge of how things actually work)
However... if you really want to disregard this advice and max out with as much memory as possible, go to OSH Park and order Frank's
memoryboard, and get six 23LC1024 chips and the other misc parts it needs. That will give you RAM for a 9 second delay. It'll also leave the 8 pin location on the audio board unused. Then your can choose either a Winbond W25Q128FV (the largest easy-to-solder) or a Micron N25Q512A13G (the largest 8 pin possible-to-solder) chip for the bottom of the audio shield.
Or if you *really* want to max things out, get Onehorse's 2Gb board. It'll need fiddling with wires to get hooked up to the right pins, but that will give you absolutely the largest NOR flash memory chip that exists in the world today!
