Best Method for Fast Playing of Samples with 8-voice Polyphony?

[Cuolin]

I have a large number of wav files sampled from a Piano that comprises 2 octave's worth of the standard 12 pitches AND 6 different velocity settings for each pitch

This means I have 2*12*6 = 144 different wav files

I want to use a capacitive touch / pressure sensitive keyboard I built to play them with 8-note polyphony from a Teensy which could also then add effects such as reverb. Ideally the latency is very low which makes me think I should load all wav files from an SD card into RAM at startup and then play from there using the Audio System Design Tool's "playMem" function once the keyboard is pressed.

The problem is that the samples are 8 seconds long and thus about 1.4MB which means that I would need ~202MB (144*1.4) of memory to hold it all. Since the Teensy 4.1 can only be expanded to include two 8MB PSRAM chips, that total extra 16MB will not be anywhere near enough space unfortunately.

Another option might be to add a 256MB Flash chip to the Teensy 4.1 which should have enough space and then use the "playFlash" function to play the sound files. My main questions are:

1) Is playing from memory with "playMem" actually significantly faster than from flash with "playFlash"? Does anyone know the % speed up?

2) Is there anything wrong about my above assumptions? Can I expand PSRAM beyond 16MB total or expand Flash beyond 256MB total?

3) The fall back plan would be to just play from an SD card with either "playSDWav" or "playSDRaw" which would solve all space problems since they would only be loaded into memory until I press on the keyboard but I am worried about both the added latency and polyphony constraints which I read were an issue with playing from an SD card. Still, I've read the SD card code has been improved over time. Has anyone recently found these constraints to be both real and problematic?


Thanks for any advice!

[Frank B]

https://github.com/FrankBoesing/Teensy-WavePlayer can do it, from SD or wherever you want.
If SD, you'll need a fast SD card. I recommend a Kingston GO! Plus >= 64GB.
Sandisk is slower.

[Frank B]

P.s. use a Teensy 4.1 and the on board SD slot. Not the one on the Audio shield.

[AlanK]

Hi,
Flash chips sold as 256Mb are 256 mega bits - you would need several chips. It's possible, but try the much easier SD card approach first. The latency is determined by the buffer size used by the audio library (which can be altered - but not if you want USB audio output) - not by the storage media. The storage media (and the buffer size) will limit the number of samples you can play simultaneously without glitches.
All the best,
Alan

[defragster]

There are 2Gbit NAND flash chips that fit the T_4.1 bottom QSPI pads.
Not sure about access rate for 8 random files?, but supported by LittleFS ... 265289728, 2000, 15000}, //Winbond W25N02G

[Frank B]

There are 2Gbit NAND flash chips that fit the T_4.1 bottom QSPI pads.
Not sure about access rate for 8 random files?, but supported by LittleFS ... 265289728, 2000, 15000}, //Winbond W25N02G

Is the issue solved that openening a spiffs file takes dozens of ms?

[AlanK]

Hi,
I've just tested playing simultaneous raw files (all started at the same instant - audio interrupts disabled while starting play) from an SD card (Sandisk Extreme 32GB) on a T4.1 with Teensyduino 1.56. Works fine for two files but audible clicks when playing three.
After changing AUDIO_BLOCK_SAMPLES to 256 (approximately 6 ms latency), however, I could play eight with few clicks.
So, it's probably worth experimenting with a more suitable SD card, and testing what level of latency is acceptable to you, before trying a more complex solution.
All the best,
Alan

[AlanK]

Hi,
Sorry - that was not a good test. If I try to play 8 files so they overlap, but aren't started simultaneously, then there are significant problems, even with AUDIO_BLOCK_SAMPLES set to 256. It may still be worth trying with a more suitable SD card.

Definitely possible with NOR flash (could have more than 8 voices), but not straightforward.

All the best,
Alan

[Myreauks]

There are 2Gbit NAND flash chips that fit the T_4.1 bottom QSPI pads.
Not sure about access rate for 8 random files?, but supported by LittleFS ... 265289728, 2000, 15000}, //Winbond W25N02G

I am also highly interested in playing multiple audio files from NAND flash simultaneously. Have any tests been conducted on this? I have a 1GB Winbond chip waiting to be soldered on my Teensy 4.1 but haven't gotten around to testing it yet. How would one transfer audio files to the chip after it is soldered to Teensy?

[Cuolin]

https://github.com/FrankBoesing/Teensy-WavePlayer can do it, from SD or wherever you want.
If SD, you'll need a fast SD card. I recommend a Kingston GO! Plus >= 64GB.
Sandisk is slower.

Thanks for your response, I will try it out when the 4.1 gets here! Others in this thread have mentioned issues with playing multiple files at the same time from SD cards. Does your specific library fix that? If so do you know what the max polyphony/files playing at the same time is?

[Frank B]

Thanks for your response, I will try it out when the 4.1 gets here! Others in this thread have mentioned issues with playing multiple files at the same time from SD cards. Does your specific library fix that? If so do you know what the max polyphony/files playing at the same time is?

This example plays 14 files : https://github.com/FrankBoesing/Teen...er_14Files.ino
However, they do not start at the same time. Did not try that. I don't think it would work with the example code, because the waveheaders need to be read when opening the files - at this takes a little time.

In your special case, I'd use "raw" files if possible. And perhaps open all of them (an array of "file", and loop over it to open all?) to save the time for reading the FAT entries and later use play( file..) and rewind().

[Frank B]

Is it possible to download your samples somewhere?
I'd like to write a MIDI example... can borrow a midi keyboard from my son.

[Cuolin]

Is it possible to download your samples somewhere?
I'd like to write a MIDI example... can borrow a midi keyboard from my son.

Yes! Here is a link to download some high quality piano samples that are also big like mine: https://www.samplerbox.org/files/ins...GrandPiano.rar

[Frank B]

Thanks! Meanwhile I found a set with 260 samples, too: https://theremin.music.uiowa.edu/MISpiano.html

I'll try aiff first, maybe it works... if not, I'll convert to raw.

[Frank B]

Thanks! Meanwhile I found a set with 260 samples, too: https://theremin.music.uiowa.edu/MISpiano.html

I'll try aiff first, maybe it works... if not, I'll convert to raw.

Had to process the sample with ffmpeg first, for a higher volume and to remove the silence at start and end... but .. i think it works good with simply using aiff files.
It uses 14 players stereo "voices", (so, 28 channels ) if more voices are needed, the oldest gets replaced.
Sounds OK for me - but I'll ask my son first, this evening... I'm not a musician.

[AlanK]

Hi,
One thing to consider is whether you want to be able to have the same file playing more than once at the same time (e.g., if you press a key repeatedly). Some approaches may not work if you do.
All the best,
Alan

[Frank B]

Hi,
One thing to consider is whether you want to be able to have the same file playing more than once at the same time (e.g., if you press a key repeatedly). Some approaches may not work if you do.
All the best,
Alan

This works - may not be desirable for a piano, but may be for other instruments.
I have uploaded the example, along with 260 high quality stero samples from a "Steinway & Sons model B".
It's a simple example of the waveplayer - you'll probably want to add pedal operation, etc. These things are beyond the scope of a simple audio object example.
I tried to keep the code simple (and i'm not a musician).
Remember to use a FAST card, and a T4.1 (because of the higher speed SD)

https://github.com/FrankBoesing/Teen...ayer_MidiPiano

(However, if you extend it - I'll be happy to add your code the examples)

[AlanK]

Hi,
I've been doing some tests using WavePlayer, with 8 identical wav files stored on an SD card. Each contains a one second long 400 Hz tone, with an amplitude of 0.12 (generated using Audacity), so each starts at zero and ends at zero. My current code allows up to 32 sounds to be played at once.

In one test, I've got it to play just two files, with a delay of roughly 20 ms between the start of each one, and recorded the USB audio output. It repeats the play every 1.5 s. So, each play should see one sinusoid start, then change amplitude after roughly 20 ms when the second starts, and then change back roughly 20 ms before the end. Each file stops with the output going to zero, so there should be no jumps in the signal.

This shows the start of one play

- the delay between the start of one file playing and the start of the second playing is roughly 20 ms as expected. Here's the end of that play, however

The drop in amplitude is roughly 95 ms from the end, and there is a noticeable (and audible) jump in the signal at that point. There are no other visible issues.
The end of each play is inconsistent - for example, in this case there are two jumps - one at the end and one just 3 ms.


Here's the code I'm using. It uses a 32 input version of mixer.h.

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>
#include <play_wav.h>
#include "MyMixer.h"


AudioPlayWav           playSdWav[32];
AudioMixer32            mixer1;
AudioOutputI2S           i2s1;
AudioOutputUSB           usb1;
AudioConnection*         patchCordToMixer[32];
AudioConnection          patchCord1(mixer1, 0, usb1, 0);
AudioConnection          patchCord2(mixer1, 0, usb1, 1);
AudioConnection          patchCord3(mixer1, 0, i2s1, 0);
AudioConnection          patchCord4(mixer1, 0, i2s1, 1);


const char *filename[32] = {
  "Test2/Test1.wav",  "Test2/Test2.wav", "Test2/Test3.wav", "Test2/Test4.wav",
  "Test2/Test5.wav",  "Test2/Test6.wav", "Test2/Test7.wav", "Test2/Test8.wav",
  "Test2/Test1.wav",  "Test2/Test2.wav", "Test2/Test3.wav", "Test2/Test4.wav",
  "Test2/Test5.wav",  "Test2/Test6.wav", "Test2/Test7.wav", "Test2/Test8.wav",
  "Test2/Test1.wav",  "Test2/Test2.wav", "Test2/Test3.wav", "Test2/Test4.wav",
  "Test2/Test5.wav",  "Test2/Test6.wav", "Test2/Test7.wav", "Test2/Test8.wav",
  "Test2/Test1.wav",  "Test2/Test2.wav", "Test2/Test3.wav", "Test2/Test4.wav",
  "Test2/Test5.wav",  "Test2/Test6.wav", "Test2/Test7.wav", "Test2/Test8.wav",
};


void setup() {
  for (int i = 0; i<32; i++)
  {
    patchCordToMixer[i] = new AudioConnection(playSdWav[i], 0, mixer1, i);
    mixer1.gain(i,0.25);
  }  
  SD.begin(BUILTIN_SDCARD);
  for (int i = 0; i<32; i++)
  {
    playSdWav[i].play(filename[i],true,true); 
  }
  AudioMemory(100);
  delay(20000); //Gives me time to start Audacity to record the output
}


void loop() {
  for (int i = 0; i<2; i++)
  {
    playSdWav[i].pause(false);
    delay(20);
  }
 delay(1500);
}

Is there something wrong with my code? The results suggest that the files aren't being played to the end.

All the best,

Alan

[Frank B]

Quite possible that you found a bug. I did not notice it so far. But keep the 128 Sample blocks in mind. A player can't start and stop in the middle of a block.
Howver ..hmm 95ms is almost in the middle of the file.. that's a really long time... i can imagine problems near the end.. within the last 3-6ms.. but 95ms?

Edit: Ok, 95 / 32 is 2.9 - a block length...

[Frank B]

The amplitude can be a mixer problem, too. The player does not touch the samples- It copies them to the output.. it may skip some, yes, if there is a bug. but it doesn't touch the amplitude.

[AlanK]

Hi,

I tested the same thing with AudioPlaySdWav objects instead. The results were very close to what I would have expected -consistent changes in amplitude close to 20 ms from the end of each play, and no jumps in the signals,

All the best,

Alan

[Frank B]

I guess if there is a problem, it should be there with a single file, too.
Can you test this?

[Frank B]

Opened an issue here: https://github.com/FrankBoesing/Teen...layer/issues/8
Will look at this when I have time (and the mood). PRs welcome.

For the Piano, it's not an issue as the samples very silent near EOF and it's not audible. Let's not pollute Cuolins thread.

[Frank B]

Well, i found an issue when playing a single file, and fixed that.
I could not reproduce the other issue, AlanK. I'd need a program (full code, without other extensions like special mixers) that I can run here. Until then, I'll close the Github issue.

[AlanK]

Hi Frank,

Sorry for the delay - I only get short opportunities with work to look at this.

Here's simpler code that just plays one file. Sometimes, it is not playing to the end - similar problem to what I reported already.

#include <Audio.h>#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>
#include <play_wav.h>


AudioPlayWav           playSdWav[4];
AudioMixer4            mixer1;
AudioOutputI2S         i2s1;
AudioOutputUSB         usb1;
AudioConnection*       patchCordToMixer[4];
AudioConnection        patchCord1(mixer1, 0, usb1, 0);
AudioConnection        patchCord2(mixer1, 0, usb1, 1);
AudioConnection        patchCord3(mixer1, 0, i2s1, 0);
AudioConnection        patchCord4(mixer1, 0, i2s1, 1);


const char *filename[4] = {
  "Test2/Test1.wav",  "Test2/Test2.wav", "Test2/Test3.wav", "Test2/Test4.wav"
};


void setup() {
  for (int i = 0; i<4; i++)
  {
    patchCordToMixer[i] = new AudioConnection(playSdWav[i], 0, mixer1, i);
    mixer1.gain(i,1.0);
  }  
  SD.begin(BUILTIN_SDCARD);
  for (int i = 0; i<4; i++)
  {
    playSdWav[i].play(filename[i],true,true); 
  }
  AudioMemory(440);
  delay(20000);
}


void loop() {
  for (int i = 0; i<1; i++)
  {
    playSdWav[i].pause(false);
  }
 delay(1500);
}

This version doesn't have the same problem. In this case, there's only one AudioPlayWav object.

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>
#include <play_wav.h>


AudioPlayWav           playSdWav;
AudioMixer4            mixer1;
AudioOutputI2S         i2s1;
AudioOutputUSB         usb1;
AudioConnection        patchCord1(mixer1, 0, usb1, 0);
AudioConnection        patchCord2(mixer1, 0, usb1, 1);
AudioConnection        patchCord3(mixer1, 0, i2s1, 0);
AudioConnection        patchCord4(mixer1, 0, i2s1, 1);
AudioConnection*       patchCord5;


const char *filename[4] = {
  "Test2/Test1.wav",  "Test2/Test2.wav", "Test2/Test3.wav", "Test2/Test4.wav"
};


void setup() {
  patchCord5 = new AudioConnection(playSdWav, 0, mixer1, 0);
  for (int i = 0; i<4; i++)
  {
    mixer1.gain(i,1.0);
  }  
  SD.begin(BUILTIN_SDCARD);
  playSdWav.play(filename[0],true,true); 
  AudioMemory(440);
  delay(20000);
}


void loop() {
 playSdWav.pause(false);
 delay(1500);
}

I've also tested with the wav file changed to just 20 ms of a 400 Hz tone, and reduced the delay to 30 ms. Here's a sample of output from the first code

and here's a sample from the second:


I hope that helps,

All the best,

Alan