changing pitch of audio samples - TeensyVariablePlayback library

This is repeatable. I hear this click at the end with a specific kick drum wav sample. https://gist.github.com/grayxr/3ca01ce5db5761a499ba554b4be84b0e (using the BD.WAV sample from the google drive link I shared above) demonstrates it. Careful, I set the sketch volume high so it's audible, but if you have equipment to record the audio and look at the waveform data, that might be preferable.
positionhigh said:
Sorry if i asked this previously @h4yn0nnym0u5e, i made a new attempt to use your modifications by dropping out temporarily other CPU and RAM hungry code to see how far i can get. I noticed that some short percussion samples end with a (full volume) click while others do not. This behaviour is consistent, it is always the same set of samples making a click while the others have no issue. All samples are faded out to zero and strangely this only happens when playing from SD Card. From Progmem (with same data) it makes no clicks.

I played a lot with the buffer values but that does not seem to make any difference in this regard. I still can't really use your wonderful work (most probably of other issues) but i am wondering if this is somehow known to you or others.
Click to expand...
 
Just reporting in... I think there are three issues for me to attack, probably in this order:
  • triggering playback from uClock or other interrupt code
  • slow playback start
  • your repeatable click
I've been struggling a bit today with getting playback to execute via EventResponder. The reason I'd like to do this is that I think it may help with interrupt triggering, but not 100% sure. At the moment I can't even make it work with immediate triggering, which ought to be a very convoluted way of just calling playWav(). Sigh. Also, I think it may be academic in the long run, as even if I succeed it's going to mean memory allocation / deallocation takes place in an ISR, which I fear is doomed to cause instability.

I can see at least one reason for the slow playback start: the file is open()ed twice when playWav() is called: once to parse the WAV header, then it's closed, and re-opened to load buffers and start playback. This is clearly the Wrong Thing to do, but I think I can see how to fix it. For now it's just another inefficiency.

I'm currently assigning lowest priority to fixing the click, simply because I'm sure it is fixable, but doing so doesn't really advance us towards the goal of tight timing for multiple tracks, and finding the issue may be pretty time-consuming.


If you can try out the concept of pre-loading an idle AudioPlaySdResmp object in the foreground, then simply setting its playback speed in the ISR, that'd be good. It may mean I can discard the effort for direct-from-ISR triggering (which is quite possibly doomed to failure anyway because of the SD access and memory allocation issues). In any case you'll probably want it for low-latency reasons - everything cued up should start withing 2.9ms of being triggered, plus hardware buffering delays.
 
I agree with your prioritization, and thank you so much for dedicating time to this!

I've switched from using SW SPI to HW SPI for talking to the OLED display, and it is way faster, and this basically eliminated the stuttering in the audio / main loop code, so if you'd like to deprioritize the interrupt capabilities, that'd be fine by me since I think I can be fine with out it.

Now, I think the biggest obstacle for me is the slow playback start. I definitely can tell timing is affected enough to make everything sound slightly off. The audible click stuff can be compensated pretty easily for with an envelope in the interim before a real fix is found for that.

But since you've asked me to test the cue/preloading stuff, I will try to get that done shortly!
 
Can you give this update a go? It's a bit of a hasty update, and it's quite likely that I've broken all the other filesystems (LittleFS etc.), but SD seems to be working and faster (~8ms/file rather than 12ms). You'll get a bit of extra speed by not having your samples buried 3 folders deep, too.

There's a limit to how fast this can get, because audio updates are taking about 40% of the CPU, which is making what should be a 5ms open and load time into 8ms. Not much to be done about that.
 
Update mentioned above now has a fix for the glitch, too. This is not production-ready - I need to remove some debug code and check whether I've broken filesystems other than SD.
 
@h4yn0nnym0u5e - I know it's a WIP, but I've tried switching to your branch code... and I'm wrestling with some of the changes to how loop_start works... I basically need a getPosition() that returns whatever value is expected by setLoopStart(), whether that's samples or blocks or millis, etc. Unless I'm missing something... is it already millis? I'm testing with stereo files...
EDIT: I didn't have that recent commit, hold up
 
@h4yn0nnym0u5e in regards to the slow playback start and the click sound -- on my end these are a million times better! I don't hear a click sound at all. I also took your advice and put my sample folder at the top level, and that plus your other changes seem to be working out very well timing wise; I'm very happy right now with where this all stands! I'm pretty much beaming right now! :D

I still owe you a sketch, I will try to get that done tonight.
 
@h4yn0nnym0u5e I have a few lingering questions as well:

1. What does adjusting the RESAMPLE_BUFFER_COUNT actually affect? I'm curious if I should be increasing this value since I'm opening lots of files concurrently given the multi-track sample playback. What are the benefits/penalties if I raise/lower this value?

2. Would using AudioPlaySdResmp.playRaw() offer better performance / lower latency?

3. It occurred to me that my hardware design for my groovebox employs a SD card extender cable so that I can expose the SD card slot through the enclosure. Could this be introducing extra latency as well? See the image below.

Screenshot 2024-09-20 at 1.01.23 PM.png
 
  1. RESAMPLE_BUFFER_COUNT affects the number of buffers allocated to each playback object, so 2048, 7 gives you seven buffers of 2048 samples, thus 28kB of memory used to buffer about 325ms of audio. Once a buffer's data is consumed the EventResponder recycles it to the back of the 7-element queue and puts the next 2k samples into it from the file. If you have 16 playback objects then that'd be 448kB of RAM used, if all tracks are playing at once. Or 896kB if you have two objects per track - hence the need for PSRAM. I'd say 3 buffers is the bare minimum, one playing, one ready, one awaiting refilling. It gets worse for playback rates over x1.0 because you use the buffers faster. A silent objects uses no memory, because it's returned to the heap - I kept this scheme, even though it results in heap fragmentation...
  2. No, RAW files are no more efficient, you still have to read them from the filesystem. They were a bit faster to start, before put in the fix to only open the file once. And they mean you have to inform the system how many tracks they contain, e.g. by keeping all your mono files in one folder and the stereo ones in another, or some other scheme.
  3. It shouldn't make a difference, unless there's some clever adaptive code under the hood which detects errors and slow the SDIO bus down, or something.
 
Here's a slightly better version of your test sketch.
  • ignores whether an object is playing(), instead uses the opposite object to the one last cued
  • doesn't silence the other object's mixer channel when triggering a cued object
It's not quite right, I think there's issues with first and last steps. It's a little bit confusing that you cue by track number, but trigger by voice number, as there just so happens to be one voice per track. Maybe cueSamples() should cue up SampleVoices, and trigger() should just iterate over the voices to see which ones are cued. You'd need a third state for cued: A, B, and Neither.
 
Alright, I fixed the sketch. I was being sloppy before. Updated sketch here.

Very cool that this works! I will have to try and benchmark this approach for my project to see if offers better timing / value, because I could see myself coming back to this a bit later and being confused on how it works.

Thanks again! I'll try to post a link to a video later demoing the progress.
 
Another question: I've noticed severe lag / program failure when trying to read/write from the SD card while these AudioPlaySdResmp are playing, which makes sense if the SD card is busy streaming audio files. Is there a simple way to set a flag to block the audio buffering from the SD card while I perform some other SD card operation, or can I just do it from an interrupt?

Maybe I will move to storing/managing project data on a non volatile flash chip (like a Winbond W25N02KVZEIR) and use LittleFS to read/write project data while audio is playing. I assume that shouldn't cause an issue.
 
Never ever access the SD card from interrupt! It's the major cause of pretty much all previous issues with SD streaming...

I would expect SD accesses to work during playback, with the caveat that the audio updates are taking ~50% of the CPU, so that will slow you down, and if you do something excessively time-consuming, then the EventResponder can't fire and re-load the buffers, so playback will stutter / glitch. There's no flag you can set as such, but if loop() doesn't exit and you don't call either yield() or delay(), you have all the SD card bandwidth.

As ever with SD access, you should aim to read and write in big chunks - at least 512 bytes, and ideally 4kB. Lots of little accesses and calls to seek() will undoubtedly risk being laggy. I wouldn't expect failure, though.


In other news, I've pushed up a new set of commits which have had LittleFS and memory streaming tested. LittleFS and SerialFlash both needed the code revised, but I haven't got SerialFlash so I couldn't test it. I've also (I believe) removed all the debug pin toggles.
 
For anyone lurking who wants to try a simple example of what we've been wittering on about these last few posts ... here's that example:

The main sketch simple.ino:
C++: 
/*
 * Demonstration of interrupt-based triggering of prebuffered
 * playback object, using the TeensyVariablePlayback library
 *
 * Uses samples converted from https://www.indiedrums.com/2012/10/10/free-drums-samples-rock-kit/
 */
#include <Audio.h>
#include <TeensyVariablePlayback.h>
#include "simple.h"

// GUItool: begin automatically generated code
AudioPlayWAVstereo       playWAVstereo1A; //xy=170,66
AudioPlayWAVstereo       playWAVstereo1B; //xy=174,108
AudioPlayWAVstereo       playWAVstereo2A; //xy=190,240
AudioPlayWAVstereo       playWAVstereo2B; //xy=194,282
AudioPlayWAVstereo       playWAVstereo3A; //xy=202,409
AudioPlayWAVstereo       playWAVstereo3B; //xy=206,451
AudioPlayWAVstereo       playWAVstereo4A; //xy=213,567
AudioPlayWAVstereo       playWAVstereo4B; //xy=217,609
AudioMixer4              mixer1L;         //xy=400,70
AudioMixer4              mixer1R; //xy=406,137
AudioMixer4              mixer2L; //xy=420,244
AudioMixer4              mixer2R; //xy=426,311
AudioMixer4              mixer3L; //xy=432,413
AudioMixer4              mixer3R; //xy=438,480
AudioMixer4              mixer4L; //xy=441,566
AudioMixer4              mixer4R;  //xy=449,638
AudioMixer4              mixerL; //xy=693,341
AudioMixer4              mixerR; //xy=702,414
AudioOutputI2S           i2sOut;         //xy=854,349

AudioConnection          patchCord1(playWAVstereo1A, 0, mixer1L, 0);
AudioConnection          patchCord2(playWAVstereo1A, 1, mixer1R, 0);
AudioConnection          patchCord3(playWAVstereo1B, 0, mixer1L, 1);
AudioConnection          patchCord4(playWAVstereo1B, 1, mixer1R, 1);
AudioConnection          patchCord5(playWAVstereo2A, 0, mixer2L, 0);
AudioConnection          patchCord6(playWAVstereo2A, 1, mixer2R, 0);
AudioConnection          patchCord7(playWAVstereo2B, 0, mixer2L, 1);
AudioConnection          patchCord8(playWAVstereo2B, 1, mixer2R, 1);
AudioConnection          patchCord9(playWAVstereo3A, 0, mixer3L, 0);
AudioConnection          patchCord10(playWAVstereo3A, 1, mixer3R, 0);
AudioConnection          patchCord11(playWAVstereo3B, 0, mixer3L, 1);
AudioConnection          patchCord12(playWAVstereo3B, 1, mixer3R, 1);
AudioConnection          patchCord13(playWAVstereo4A, 0, mixer4L, 0);
AudioConnection          patchCord14(playWAVstereo4A, 1, mixer4R, 0);
AudioConnection          patchCord15(playWAVstereo4B, 0, mixer4L, 1);
AudioConnection          patchCord16(playWAVstereo4B, 1, mixer4R, 1);
AudioConnection          patchCord17(mixer1L, 0, mixerL, 0);
AudioConnection          patchCord18(mixer1R, 0, mixerR, 0);
AudioConnection          patchCord19(mixer2L, 0, mixerL, 1);
AudioConnection          patchCord20(mixer2R, 0, mixerR, 1);
AudioConnection          patchCord21(mixer3L, 0, mixerL, 2);
AudioConnection          patchCord22(mixer3R, 0, mixerR, 2);
AudioConnection          patchCord23(mixer4L, 0, mixerL, 3);
AudioConnection          patchCord24(mixer4R, 0, mixerR, 3);
AudioConnection          patchCord25(mixerL, 0, i2sOut, 0);
AudioConnection          patchCord26(mixerR, 0, i2sOut, 1);

AudioControlSGTL5000     sgtl5000;       //xy=842,409
// GUItool: end automatically generated code

#define COUNT_OF(a) ((int)(sizeof a / sizeof a[0]))

//==================================================================
SampleVoice sampleVoices[] = {
  {{{playWAVstereo1A}, {playWAVstereo1B}}, mixer1L, mixer1R, -1},
  {{{playWAVstereo2A}, {playWAVstereo2B}}, mixer2L, mixer2R, -1},
  {{{playWAVstereo3A}, {playWAVstereo3B}}, mixer3L, mixer3R, -1},
  {{{playWAVstereo4A}, {playWAVstereo4B}}, mixer4L, mixer4R, -1}
};


/*
 * Cue up a voice ready to be triggered by IntervalTimer. Can take a
 * while to pre-load the buffers, but it's not time-critical so long
 * as it's ready when the step is triggered.
 */
char cueVoice(SampleVoice& voice, const char* file, float rate)
{
  char result = ' ';
  if (voice.cued < 0) // not cued, valid to cue up
  {
    int toCue = -1;

    // pick a player to use
    if (!voice.players[0].player.isPlaying()) // pick player A if it's idle...
      toCue = 0;
    else if (!voice.players[1].player.isPlaying()) // ...or player B if that's idle...
      toCue = 1;
    else if (voice.players[0].started < voice.players[1].started) // ... or A if it started longer ago...
      toCue = 0;
    else            // ... or B
      toCue = 1;
    result=toCue?'B':'A';
   
    // we have a player: cue it up
    voice.players[toCue].player.setPlaybackRate(0.0f); // don't output yet
    voice.players[toCue].player.playWav(file);  // get it pre-loaded
    voice.players[toCue].started = millis(); // it was ready at this time
    voice.rate = rate;  // rate to start at when triggered
    voice.cued = toCue; // signal it's ready
  }

  return result; // which sample player got used, A or B, or space if neither
}

/*
 * Called from IntervalTimer callback to start voice
 * playing at a precise instant.
 */
void startVoice(SampleVoice& voice)
{
  if (voice.cued >= 0) // safety check
  {
    voice.players[voice.cued].player.setPlaybackRate(voice.rate);
    voice.cued = -1; // no longer cued
  }
}


void initVoices(void)
{
  for (int i=0;i<COUNT_OF(sampleVoices);i++)
  {
    for (int j=0;j<COUNT_OF(sampleVoices[i].players);j++)
      sampleVoices[i].players[j].player.setBufferInPSRAM(true);
    mixerL.gain(i,0.2f + 0.2f*i);    
    mixerR.gain(i,0.8f - 0.2f*i);    
  }
}
//==================================================================
IntervalTimer intvl;
volatile int stepNum;
void intvlFn(void)
{
  digitalToggleFast(LED_BUILTIN);
 
  // start all cued samples
  for (int i=0;i<COUNT_OF(sampleVoices);i++)
    if (sampleVoices[i].cued >= 0)
      startVoice(sampleVoices[i]);

  // signal foreground that next step can be cued up    
  stepNum++;
  if (stepNum >= 16)
    stepNum = 0;
}

//==================================================================
void setup()
{
  AudioMemory(40);

  while (!Serial)
    ;
   
  pinMode(LED_BUILTIN, OUTPUT);

  // initialise output hardware
  sgtl5000.enable();
  sgtl5000.volume(0.15);

  // initialise SD card
  while (!(SD.begin(BUILTIN_SDCARD)))
  {
    // print a message repeatedly until begin() succeeds
    Serial.println("Unable to access the SD card");
    delay(500);
  }
 
  initVoices();
  Serial.println("Running");  

  intvl.begin(intvlFn, 125000); // trigger at 125ms interval
}


int lastStepNum = -1;
void loop()
{
  if (lastStepNum != stepNum)
  {
    char cued = ' ';
    lastStepNum = stepNum;

    switch (lastStepNum)
    {
      case 0:
      case 7:
      case 8:
        cued = cueVoice(sampleVoices[0], "/drums/44k-Kick-ff-2.wav", 1.0f);
        break;
       
      case  2:
      case  6:
      case 11:
      case 14:
        cued = cueVoice(sampleVoices[3], "/drums/44k-HiHat-Tip-2.wav", 1.0f);
        break;

      case  4:
      case 12:
        cued = cueVoice(sampleVoices[1], "/drums/44k-Snare-ff-1.wav", 1.0f);
        break;
       
      case 10:
        cued = cueVoice(sampleVoices[3], "/drums/44k-HiHat-Open.wav", 1.0f);
        break;
    }

    if (' ' != cued)
      Serial.printf("%lu: Cue step %d: voice %c\n", millis(), lastStepNum, cued);
  }
}

Sketch header file simple.h:
C++: 
#define AudioPlayWAVstereo AudioPlaySdResmp // map to different object

struct Player
{
  AudioPlayWAVstereo& player;
  uint32_t started;
};

struct SampleVoice
{
  Player players[2]; AudioMixer4 mixerL, mixerR; int cued; float rate;
};
Put these together in a folder called simple, so the Arduino IDE can load them:
1727090665714.png


A 4-sample drum kit in the attached drums.zip - put these files on your SD card in a /drums folder.

Get a copy of the library with the required changes - currently here but subsequent updates may be on a different branch. The default buffers aren't big enough (I didn't change Nic's values), so you will need to edit ResamplingSdReader.h - these settings seem OK:
C++: 
#define RESAMPLE_BUFFER_SAMPLE_SIZE 1024
#define RESAMPLE_BUFFER_COUNT           5
 

Attachments

  • drums.zip
    203.9 KB · Views: 20
Thanks for all your help! Would you prefer I start a new thread about the concurrent playback + SD card access? I’m trying to read/write my custom data structures in chunks to the SD card while the buffered audio is playing and I can’t seem to get past it hanging.
 
No worries.

Might be a good idea to start a new thread, I guess. If you link to it from here then the assiduous reader will find it if they need it!
 
Nevermind, I figured out that I just needed to implement non-blocking buffered SD reads/writes from the main loop. Before, I was just doing large blocking read/write calls, and then I was trying to do buffered read/writes from within a while loop while the files remained open (which I didn't realize defeated the purpose for the buffering). I have it working now! I hope reading/writing 40kb chunks at a time is not too large, but seems to be working for me so far.
 
40kB should be fine, though I'd always recommend instrumenting your code to see how much margin you have.

Yes, it's easy to lose track of the fact that you somehow have to allow the buffering to take place. I quite like the following structure for "real" sketches:
C++: 
void setup()
{
    initThis();
    initThat();
    initTother();
}

void loop()
{
    updateThis();
    updateThat();
    updateTother();
}

Then when you write the update???() functions, you're less likely to lose sight of the fact that they have to exit to allow each other to execute.

If you absolutely must complete a time-consuming process within an update???(), putting in strategically-placed yield() calls will allow the buffer re-loads to occur.
 
Got it. I take it back though, I was able to get my project file _reads_ to happen concurrently with buffered SD playback using a non-blocking main loop approach, but as far as writes go, even in 512 byte sized chunks, it seems it pretty much halts the SD card and the SD buffered audio stops. Here is the separate thread.
 
@h4yn0nnym0u5e I'm also interested in using sample data to display the sample waveform on my display. Is there currently a way I can read out the buffered sample data in a loop or something? I wasn't sure of a method on the playback object to fetch the sample data. Just curious.
 
You must log in or register to reply here.
Back
Top