How to load WAV from SD Card to RAM for AudioPlayMemory for fast playing/polyphony?

D

DGStorm

New member
I'm using a Teensy 4.1 with an Audio Board (Teensyduino 1.57) to play instruments from sample files on a SD Card with a very low latency (<10ms) for live playing and polyphony. There will be a knob that allows you to select an instrument and then the program will load that instrument's samples from the SD Card for playing.

I've tested the latency for playing a sine wave with the Audio library and then listening with an electret mic for the note-to-sound latency and it is around 7 ms. I'd like to play the WAV file samples on the SD card just as fast, but running into problems. For reference, I'm using this SD card which I think should be fast enough: https://www.amazon.com/gp/product/B09X7CRKRZ/

I first tried to use the AudioPlaySdWav from the Audio library which clocks in around 40 ms, slower than I need. I could use wav2sketch to turn the WAV files into .h and .cpp sketch files which I guess then get stored on Flash memory and then use AudioPlayMemory to play them but I have many gigabytes of audio files on the SD Card and they cannot all be stored in Flash.

My Teensy has two chips soldered to the underside for extra 16MB of PSRAM so I could try to load just the currently selected instrument's worth of WAV files from the SD Card into an array and then use AudioPlayMemory to play the arrays as needed, I figure this is the fastest possible way too since they would be playing straight from RAM. Does anyone know if there are libraries that allow you to quickly load a WAV from an SD Card into an array in PSRAM memory?

Also does anyone know any better or faster ways to do this?


Side note: I've also been trying to use a Raspberry PI 4 for this since that has tons of RAM to load the WAV files into memory, but I found the latency to be around 60 ms using the code from the Samplerbox project: https://github.com/josephernest/SamplerBox

Has anyone been able to get a Raspberry Pi or other set up around 10 ms latency?

Thanks for any advice!
 
h4yn0nnym0u5e said:
Try this: https://forum.pjrc.com/threads/70963-Yet-Another-File-Player-(and-recorder) ? In particular the SDPiano demo does pretty much the sort of thing you’re describing, I think. I’m working on building in the low-latency pre-buffering, but it’s not working properly yet.
Click to expand...

Thanks very much, this library was able to get my latency down to ~12ms which is acceptable for my use case and the first time I've been able to get that low with samples!

A couple questions I had:

What is the difference between the "low-latency pre buffering" you mentioned above and what the library currently does with putting the beginning of the file in memory, is this a way of making it even faster?

If I have extra PSRAM chips soldered on like I mentioned, how might I utilize that to load a larger initial slice of the WAV file in memory or even load the whole wav file in memory for fast playing? I noticed a few variables that sound like they pertain to this but between the following 3 variables (or another variable I'm not seeing), I'm not sure what to change:

1) CHUNKLEN (set to 512 in SDpiano.ino)​
2) STARTLEN (set to 60*512 in SDpiano.h)​
3) The 32768 value in this line of code in SDpiano.ino: pv.playWAVstereo1.createBuffer(32768,AudioBuffer::inExt);​



Thanks again!
 
DGStorm said:
Thanks very much, this library was able to get my latency down to ~12ms which is acceptable for my use case and the first time I've been able to get that low with samples!

A couple questions I had:

What is the difference between the "low-latency pre buffering" you mentioned above and what the library currently does with putting the beginning of the file in memory, is this a way of making it even faster?

If I have extra PSRAM chips soldered on like I mentioned, how might I utilize that to load a larger initial slice of the WAV file in memory or even load the whole wav file in memory for fast playing? I noticed a few variables that sound like they pertain to this but between the following 3 variables (or another variable I'm not seeing), I'm not sure what to change:

1) CHUNKLEN (set to 512 in SDpiano.ino)​
2) STARTLEN (set to 60*512 in SDpiano.h)​
3) The 32768 value in this line of code in SDpiano.ino: pv.playWAVstereo1.createBuffer(32768,AudioBuffer::inExt);​



Thanks again!
Click to expand...
great that it’s working for you!

The low-latency pre-buffering is currently only in the SDPiano demo, but I’m hoping to build it in to the buffered playback library itself as it seems to be a pretty common need. So there’s no real difference and there probably won’t be any speed improvements as a result. The way to even lower latency is to reduce the audio data block size from 128 samples (2.9ms) to 64 or 32. However, a few of the audio classes don’t play nicely if you do that, so I’d advise caution.

No need to change CHUNKLEN, it only governs the size of chunks loaded for de-interleaving into two mono buffers, because the AudioPlayMemory class is mono-only.

STARTLEN governs the size of your “initial slice”, so change that to suit. For SDPiano they’re all the same size, you might want to be cleverer. I’ve not catered for the initial slice being the whole file, so it’s fairly likely to keel over if that’s the case … a bit of a re-write would be needed. I just use extmem_malloc() to allocate those buffers, so it should work the same for you - see the prepNote() function.

The createBuffer() function sets up the buffer used while streaming from SD card, so it’s continually filled from card and emptied by the audio updates. As long as it’s big enough to avoid glitches if the SD card does a slow read, bigger is not necessarily better. The 32k size I used results in each refill read being 16k; I’d probably not go below 8k buffer size unless I was super confident of the card performance. With 8M or 16M of PSRAM, a 32k buffer doesn’t make much of a dent!
 
Hey there!

I just found this thread and your libraries @PMh4yn0nnym0u5e. I've been rolling around this idea in my mind since years but never got to realize it as I was struggling to get the right components. I think your library covers everything I need (!!) but I wanted to quickly double check before I deep dig in.

Sorry to hijack the thread but this felt like the right spot to reach you (your direct message folder is full btw.).

It would be amazing if you had a moment to read this and let me know if I'm on the right track.

My idea is:
* I have a big library of cut-to-beat (mostly 4 beats) loops from all kind of songs (sampled and cut them by myself)
* I'm mostly making music on my hardware setup jamming (midi clock keeps running) so I wanted to implement the following in hardware (Teensy 4.1).
* User picks two or more samples (would add an oled and encoder control for that)
* sample is either loaded from sd to memory in the background (while the last sample still plays) or is streamed and launches instantly on the right beat.
* sample is timed to the current midi clock so it would match to four beats in length (This would be using variable rate. Its okay if the sample plays at different pitch. Maintaining pitch would be a dream, but don't really care that much)
* the user has the option to pick a different sample and it'll start playing once we're back on the first beat
* (there'd probably be more complex additions like auto slicing the sample and playing based on incoming midi notes, etc. but that's secondary)

I have pretty solid experience with all the midi, oled, encoder stuff on teensy (and daisy). The more complex audio things and the background sd card reading and streaming while keeping things realtime have always been a big wall though which I was never able to break through.

Your mentioned SDpiano example and teensy-variable-playback fork should cover my needs. What do you think? Does my idea sound reasonable with those?

Again: sorry for hijacking the thread and I really appreciate your code and input!

Thanks!
Daniel
 
I know my PM folder is full ... it's self-defence: I got a lot of questions which could / should be asked on the open forum!

With 4-beat samples and needing to stretch to fit the current MIDI clock, your only real option is TeensyVariablePlayback: this branch is the one to try out. This doesn't have the preload to RAM feature I put in my buffered playback library, and you can't easily set the playback buffer sizes - you have to edit ResamplingSdReader.h. Having said that, it defaults to 7 512-sample buffers so that should be OK for most purposes. The "only" effect of lack of preload is to make latency slightly worse, as when you call play("filename.wav") it has to open the file, parse the WAV header and load the 7k of sample buffers, which can take a few milliseconds.

Your UI seems simple enough - if the user changes a filename during the loop, the next play() call simply uses that when the loop re-starts. You might want to have two playback objects per track, and alternate them every other loop, so the tail can continue to play out while the (possibly new) head is being started. But that's an implementation detail.

The original "customer" for that branch of TeensyVariablePlayback never managed to get it to work, and didn't seem disposed to follow the Forum Rule of posting an Arduino IDE based sketch which reproduced the issues, so I let the effort fizzle out. If you use it and think you find a bug, please do post - perhaps on Nic's original thread? If it's getting some love I'll see if Nic wants to pull it into his repo, though I did make a few changes he may not have the time or inclination to review...

I really need to do a re-worked version of SDpiano which uses the preload ... it'd be much simpler! It loads the first part of every sample into PSRAM (261 of them ... should be 264 but some were missing from the sample set), starts playback of the right one depending on MIDI note / velocity, then transitions to SD card playback. Doesn't seem to have any issues with 8-note stereo polyphony.
 
Okay great! I'll hopefully have a first look this weekend and dig through your library and the variable playback thread.
Thanks for the instant response and for sharing your code and knowledge!
 
You must log in or register to reply here.
Back
Top