changing pitch of audio samples - TeensyVariablePlayback library

Interim report….

I got some incorrect output from 100% amplitude 330Hz sine waves at 2.5x playback speed, traced to overflow in the interpolation function - fix pushed.

I also tested a stereo file for the first time, and it played OK on its own, but acquired some weird high frequency component if another (mono) file was played at the same time. This needs investigation, obviously, but I’d advise testing with mono files only with the current code.
 
I only use mono samples and i think this will never change.

In the meantime i have reverted to an older version of the variable_playback before Nic introduced the additional file abstraction layer for serialflash and qspi-flash.
These more recent changes had increased CPU and/or RAM usage and the overall stability (at least in our project) was worse than it was before.

This probably ditches the new advanced crossfaded-loop-types but i think it makes much more sense getting the basic stuff working very stable before dealing with these on-top features again.

Now, with this older variable_playback library back in place (this sligthly modified version is in our repo and split into 2 parts, 1 for progmem+sd and 1 for flash only), i will try if i can retrofit your buffer-enhancements to get it going for sd-card. I am not sure if that is possible or if your code relies on the newer file abstraction but i think i will find out very soon :_)
 
That's probably just as well - there seem to be serious issues playing stereo files at 2.5x speed! We may need to involve Nic to fix these, if he can spare a bit of time, as I'm getting a bit lost.

I think you'll find most of my changes are in playresmp.h, ResamplingReader.h and IndexableFile.h. There have been minor changes to Resampling<something>Reader.h, and interpolation.cpp. Good luck!
 
I think going over 2x is too much to ask, anyway. I would be happy with a +-5 semitone pitchshift.

About the code, yes i've seen that and think that is doable but i have trouble to revert from Nics file-access abstraction layer that he put in between in the newer versions.
 
I started a new project which uses h4yn0nnym0u5e's async-read-with-interpolation-01 branch of the teensy-variable-playback code. I had asked him about it in another thread and he recommended to add questions here.
The idea is to have two+ midi synced four (8/16) beat loops playing so they line up with each others (and the rest of my music hardware). I wanna load different loops from a library and blend between them. Have had the idea for a while but now just jumping into trying this with the library.

I threw a bunch of audio loops onto the sd-card and got everything working using one AudioPlaySdResmp instance (based on the sd_play_all example in https://github.com/h4yn0nnym0u5e/te...tree/feature/async-read-with-interpolation-01). Changing time works nicely (haven't looked into the midi part yet), all my samples work and there is no audible cut between the samples.

After I use .playWav() on a second AudioPlaySdResmp instance, only the first two samples work correctly and I get errors thrown about files that worked fine before.

Any idea what I might be missing?

Here is my code (I'm using AudioOutputUSB for now)
Code:
// Plays all WAV files from the root directory on the SD card 
#include <Arduino.h>
#include <Audio.h>
#include <SD.h>
#include <TeensyVariablePlayback.h>

// GUItool: begin automatically generated code
AudioPlaySdResmp           playSd2; //xy=616,503
AudioPlaySdResmp           playSd1;     //xy=617,266
AudioMixer4              mixer2; //xy=1223,510
AudioMixer4              mixer1;         //xy=1225,419
AudioOutputUSB           usb1;           //xy=1445,474
AudioConnection          patchCord1(playSd2, 0, mixer1, 1);
AudioConnection          patchCord2(playSd2, 1, mixer2, 1);
AudioConnection          patchCord3(playSd1, 0, mixer1, 0);
AudioConnection          patchCord4(playSd1, 1, mixer2, 0);
AudioConnection          patchCord5(mixer2, 0, usb1, 1);
AudioConnection          patchCord6(mixer1, 0, usb1, 0);

/*
AudioPlaySdResmp         playSd1;     //xy=324,457
//AudioOutputI2S           i2s2;           //xy=840.8571472167969,445.5714416503906
AudioOutputUSB i2s2;
AudioConnection          patchCord1(playSd1, 0, i2s2, 0);
AudioConnection          patchCord2(playSd1, 1, i2s2, 1);
// GUItool: end automatically generated code
*/
AudioControlSGTL5000     audioShield;

#define A14 10

char** _filenames  = nullptr;
uint16_t _fileIndex = 0;
uint16_t _numWaveFiles = 0;

const int analogInPin = A14;
unsigned long lastSamplePlayed1 = 0;
unsigned long lastSamplePlayed2 = 0;

double getPlaybackRate(int16_t analog) { //analog: 0..1023
    return  (analog - 512.0) / 512.0;
}

uint16_t getNumWavFilesInDirectory(char *directory);
void populateFilenames(char *directory, char **filenames);

void setup() {
    analogReference(0);
    pinMode(analogInPin, INPUT_DISABLE);   // i.e. Analog

    Serial.begin(57600);

    if (!(SD.begin(BUILTIN_SDCARD))) {
        // stop here if no SD card, but print a message
        while (1) {
            Serial.println("Unable to access the SD card");
            delay(500);
        }
    }
    audioShield.enable();
    audioShield.volume(0.5);

    mixer1.gain(0, 1.0);
    mixer2.gain(0, 1.0);
    mixer1.gain(1, 1.0);
    mixer2.gain(1, 1.0);
    playSd1.enableInterpolation(true);
    playSd2.enableInterpolation(true);
    playSd1.setPlaybackRate(1.0);//getPlaybackRate(newsensorValue));
    playSd2.setPlaybackRate(1.0);//getPlaybackRate(newsensorValue));

    _numWaveFiles = getNumWavFilesInDirectory("/");
    Serial.printf("Num wave files: %d\n", _numWaveFiles);
    _filenames = new char*[_numWaveFiles];
    populateFilenames("/", _filenames);

    AudioMemory(24);
}

void loop() {
    unsigned currentMillis = millis();

    if (currentMillis > lastSamplePlayed1 + 1000) {
        if (!playSd1.isPlaying()) {
            
            if (playSd1.playWav(_filenames[_fileIndex])) {
              lastSamplePlayed1 = currentMillis;
              Serial.printf("playing 1 %s\n", _filenames[_fileIndex]);
            }
            _fileIndex++;
            _fileIndex %= _numWaveFiles;
        }

        if (!playSd2.isPlaying()) {
            
            if (playSd2.playWav(_filenames[_fileIndex])) {
              lastSamplePlayed2 = currentMillis;
              Serial.printf("playing 2 %s\n", _filenames[_fileIndex]);
            }
            _fileIndex++;
            _fileIndex %= _numWaveFiles;

            Serial.print("Memory: ");
            Serial.print(AudioMemoryUsage());
            Serial.print(",");
            Serial.print(AudioMemoryUsageMax());
            Serial.println();
        }
    }

    delay(10);
}

uint16_t getNumWavFilesInDirectory(char *directory) {
  File dir = SD.open(directory);
  uint16_t numWaveFiles = 0;

  while (true) { 

    File files =  dir.openNextFile();
    if (!files) {
      //If no more files, break out.
      break;
    }

    String curfile = files.name(); //put file in string
    
    int m = curfile.lastIndexOf(".WAV");
    int a = curfile.lastIndexOf(".wav");
    int underscore = curfile.indexOf("_");

    // if returned results is more then 0 add them to the list.
    if ((m > 0 || a > 0) && (underscore != 0)) {  
        numWaveFiles++;
    }
    
    files.close();
  }
  // close 
  dir.close();
  return numWaveFiles;
}

void populateFilenames(char *directory, char **filenames) {
  File dir = SD.open(directory);
  uint16_t index = 0;

  while (true) { 

    File files =  dir.openNextFile();
    if (!files) {
      //If no more files, break out.
      break;
    }

    String curfile = files.name(); //put file in string
    
    int m = curfile.lastIndexOf(".WAV");
    int a = curfile.lastIndexOf(".wav");
    int underscore = curfile.indexOf("_");

    // if returned results is more then 0 add them to the list.
    if ((m > 0 || a > 0) && (underscore != 0)) {  
        filenames[index] = new char[curfile.length()+1] {0};
        memcpy(filenames[index], curfile.c_str(), curfile.length());
        index++;
    } 
    files.close();
  }
  // close 
  dir.close();
}

namespace std {
    void __throw_bad_function_call() {}
    void __throw_length_error(char const*) {}
}

And the errors:
Code:
Num wave files: 25
playing 1 Fanta4_ZugeilfürdieseWeltwdr294BPM2.wav
Memory 1: 0,0
Not able to open file: Fanta4_Allesistneu112BPM2.wav
Memory 2: 4,6
Not able to open file: Fanta4_Die4.Dimension105BPM2.wav
Memory 2: 4,6
Not able to open file: Fanta4_Die4.Dimensionwdr105BPM2.wav
Memory 2: 4,6
playing 1 Fanta4_Die4.Dimensionwdr2105BPM2.wav
Memory 1: 4,6
playing 2 Fanta4_Ganznormal101BPM2.wav
Memory 2: 2,6
expected RIFF (was 
)
Needs 16 bit audio! Aborting.... (got 0)Memory 1: 4,8
playing 2 Fanta4_Machdichfrei84BPM2.wav
Memory 2: 2,8
playing 1 Fanta4_NeuesLand94BPM2.wav
Memory 1: 2,8
Not able to open file: Fanta4_NeuesLand294BPM2.wav
Memory 1: 4,8
Num wave files: 25
playing 1 Fanta4_ZugeilfürdieseWeltwdr294BPM2.wav
playing 2 Fanta4_Allesistneu112BPM2.wav
Memory: 0,0
playing 2 Fanta4_Die4.Dimension105BPM2.wav
Memory: 4,8
Num wave files: 25
playing 1 Fanta4_ZugeilfürdieseWeltwdr294BPM2.wav
playing 2 Fanta4_Allesistneu112BPM2.wav
Memory: 0,0
playing 2 Fanta4_Die4.Dimension105BPM2.wav
Memory: 4,8
Not able to open file: Fanta4_Die4.Dimensionwdr105BPM2.wav
Memory: 4,8
playing 2 Fanta4_Die4.Dimensionwdr2105BPM2.wav
Memory: 4,8
Not able to open file: Fanta4_Ganznormal101BPM2.wav
expected RIFF (was 
)
Needs 16 bit audio! Aborting.... (got 0)Not able to open file: Fanta4_Machdichfrei84BPM2.wav
Not able to open file: Fanta4_NeuesLand94BPM2.wav
Not able to open file: Fanta4_NeuesLand294BPM2.wav
 
Last edited:
Can't 100% reproduce this here ... but I also couldn't get it to work at all with just the USB output, I had to switch to the audio adaptor. It used to be the case USB-only didn't work, because it couldn't have "update responsibility", but Paul did a timer-based fix for that. I've never tested that it's robust, though.

It looks as if you have at least one incorrectly-formatted file, though if play fails you don't report the file name so I've no idea which one it is. That's the expected RIFF (was ) Needs 16 bit audio! Aborting.... report. The rest looks a bit like a flaky SD card. I've had good results from pretty much any SanDisk card, but poor ones from some Kingston cards. There's a card test sketch somewhere, possibly in the Examples folder. You want one that tests access speeds using multiple block sizes and file counts, as just accessing a single file gives a rather optimistic picture of card speed. A mono file needs 86kB/s, and stereo 172kB/s; the read latency figure is also important, decent cards seem to be <3ms, even when reading 16 files in 64-sector chunks.
 
Can't 100% reproduce this here ... but I also couldn't get it to work at all with just the USB output, I had to switch to the audio adaptor. It used to be the case USB-only didn't work, because it couldn't have "update responsibility", but Paul did a timer-based fix for that. I've never tested that it's robust, though.

It looks as if you have at least one incorrectly-formatted file, though if play fails you don't report the file name so I've no idea which one it is. That's the expected RIFF (was ) Needs 16 bit audio! Aborting.... report. The rest looks a bit like a flaky SD card. I've had good results from pretty much any SanDisk card, but poor ones from some Kingston cards. There's a card test sketch somewhere, possibly in the Examples folder. You want one that tests access speeds using multiple block sizes and file counts, as just accessing a single file gives a rather optimistic picture of card speed. A mono file needs 86kB/s, and stereo 172kB/s; the read latency figure is also important, decent cards seem to be <3ms, even when reading 16 files in 64-sector chunks.

The errors only occur when I use two instances. None of these errors show when using one instance - all the .wav files work then.

I'll do the card test and maybe get a different card. Was using this one: https://www.amazon.com/dp/B09WW54LLV?ref=ppx_yo2ov_dt_b_product_details&th=1

I need to do some soldering to get my audio shield mounted but will do so and report back if anything got resolved from switching to that audio out.

Thanks for your help h4yn0nnym0u5e!
 
No worries, I'd like to know this works OK...

Amazon appear to have grabbed their images straight off the PNY website, so hard to know if you got hit with a faulty or knockoff card. I tend to buy from MyMemory here in the UK, and stick to SanDisk or Samsung cards. Most purchases (pretty infrequent) are for my DSLR, and losing images is not an option, so weird names and own-brands are a no-no! I think there may be utilities which can spot a counterfeit card, and also one for the official SD formatter, though I've never noticed the latter to make any difference.
 
SD card results are looking good. This should be more than enough speed for 2-3 audio streams - right?
I guess it must be related to me using usb audio. Hopefully gonna do soldering tonight and then can test.

Code:
Reading SDTEST1.WAV & SDTEST2.WAV staggered:
  Overall speed = 1.85 Mbyte/sec
  Worst block time = 0.83 ms
    28.71% of audio frame time

Reading SDTEST1.WAV, SDTEST2.WAV, SDTEST3.WAV:
  Overall speed = 1.84 Mbyte/sec
  Worst block time = 1.25 ms
    43.05% of audio frame time

Reading SDTEST1.WAV, SDTEST2.WAV, SDTEST3.WAV staggered:
  Overall speed = 1.84 Mbyte/sec
  Worst block time = 1.25 ms
    43.05% of audio frame time

Reading SDTEST1.WAV, SDTEST2.WAV, SDTEST3.WAV, SDTEST4.WAV:
  Overall speed = 1.84 Mbyte/sec
  Worst block time = 1.66 ms
    57.32% of audio frame time

Reading SDTEST1.WAV, SDTEST2.WAV, SDTEST3.WAV, SDTEST4.WAV staggered:
  Overall speed = 1.84 Mbyte/sec
  Worst block time = 1.66 ms
    57.35% of audio frame time
 
Should be fine: you'll get an improvement if you bump the buffer sizes a bit, too, assuming you can afford the memory. Look at line 19 in ResamplingSdReader.h, change #define RESAMPLE_BUFFER_SAMPLE_SIZE 512 to 1024 or 2048. 1024 will use 14kB per playback object, for example.

I was comfortably getting 4-file playback last night, if you were to push to 2048-sample buffers I'd expect 8 files to be achievable, using 224k of RAM. Note you can use the setBufferInPSRAM() function if you have PSRAM fitted ... just now I can't remember when you use it ... probably in setup().
 
Good news: This is working great now. I'm still on usb audio but I think that the issue was that I had your main teensy-variable-playback version still in my libraries folder (together with the branch you told me to use) and that probably made the build link to the wrong one.
Now I gotta work on the actual loop syncing and won't be able to tell much about latency until I have it fully hooked up in hardware.
Thanks for your help again h4n0nnym0u5s!
 
Ah, OK, glad it's sorted out. There is looping built in to the library, but it's possibly easier to "roll your own" if you're aiming for MIDI sync. It does have some neat features like cross-fading the loop start and end, though...

Do let us know how you get on.
 
Got a question about your library h4n0nnym0u5e:

While playing (and maybe later right at the start) I want to jump to a specific position within my sample/loop. I don't think I found a function that currently offers that. Wanted to quickly double check before I add it.

Should be something like:
Code:
	bool setBufferPosition(int newBufferPosition)
	{
		if (newBufferPosition >=  _loop_finish )
			return false;
		else if (_bufferPosition1 < _header_offset)
			return false;
		else
			_bufferPosition1 = newBufferPosition;
		return true;
	}
Right?
 
Not 100% sure but I believe you’re right, no such function exists. You can already start at a specific position, by setting the loop start and using setPlayStart(play_start::play_start_loop).

Bear in mind this will potentially cause playback issues. The whole point of the modifications I did was to predict the next playback data and have it buffered ready for the audio update in good time. A full implementation would need to reload at least some of the buffers as well as changing the file position.

That would be a non-trivial task, and even then you’d probably have to accept some latency on the jump time. For even minimal safety you’d want to leave the buffer after the currently-playing one intact, and preload the rest. With the buffers at 512 samples as they are now, that’s 11.6ms per buffer, so you’d get the new audio playing sometime between 11.6 and 23.2ms after you requested the change. Or … there might be better ways. I’d have to think. And of course you’re pretty much guaranteed a click at the transition point.
 
Last edited:
Oh, and it’s not my library, it’s Nic’s! I just come on the internet and make a nuisance of myself with it…
 
Hey h4n0nnym0u5e, I was just looking back into this to see if it might work for my eurorack looper project. I cant seem to get my code to compile though. I've tried a couple of your branches, currently trying to use https://github.com/h4yn0nnym0u5e/te...tree/feature/async-read-with-interpolation-01

I keep getting an error: src/ResamplingReader.h:428: undefined reference to `fastinterpolate(short, short, short, short, float)'
Any idea why this is happening?
 
Hey h4n0nnym0u5e, I was just looking back into this to see if it might work for my eurorack looper project. I cant seem to get my code to compile though. I've tried a couple of your branches, currently trying to use https://github.com/h4yn0nnym0u5e/te...tree/feature/async-read-with-interpolation-01

I keep getting an error: src/ResamplingReader.h:428: undefined reference to `fastinterpolate(short, short, short, short, float)'
Any idea why this is happening?
Should work OK, I've had no such problem recently. Only thing I can think is either you have two copies and you're somehow getting the wrong one, or you only have the one copy but it's corrupted somehow.
 
I just tested the combo of setting loop start and using setPlayStart(play_start::play_start_loop). Unfortunately it won't play anything. My guess is that the data isn't cached yet? I was jumping to 1/2 of the sample length which is probably a lot more than what's cached. My use case / idea would be to slice the sample into 1/8th bits and then be able to launch from any of those slice points into a full loop.

On the positive side: By measuring the sample position and comparing it to the midiclock I'm now able to constantly adapt the playback rate which seems to work great (still testing on usb directly). It's funny at the start it always sounds as if it was a turntable quickly accelerating and then getting into regular speed. The interpolation and different speeds sound great!
 
I've branched again, because Nic's fixes for interpolation aren't fully merged with his master branch at the time of writing. You can find my attempts at https://github.com/h4yn0nnym0u5e/te...tree/feature/async-read-with-interpolation-01 - I've done some testing and I think stereo files, all forms of looping, dual-head playback (at loop ends) and interpolation are much improved.

That's awesome.
So in this version the buffer is created (and deleted) dynamically, compared to your buffered sample player which uses fixed size ones, am I correct?
Would it be possible to have fixed sized ones and maybe a little more persistent? like erasing the data only when the buffer is full?
Let me explain myself, lets say I have several of these playing drum samples, if one is playing a hihat in 16th notes at 120bpm, that is 125 milliseconds, which in terms of memory would be about 10k, if it's playing that sound continuously and the buffer it's already fulled with the sample data the impact on performance should be minimal.
Does this have sense? surely I'm not taking in account lots of tech aspects...
 
The original code did a lot of buffer allocation and deletion. For my branch I removed most of that, so buffers are only created when play() is called, and deleted when playback stops - in between, they're re-used for new sample data. That was a fairly small change, at the lowest level of the code (IndexableFile.h), but seemed to me a small, non-architectural change which would improve efficiency very slightly, and also make a small reduction in the likelihood of heap fragmentation.

To implement my scheme of pre-allocated buffers would require a more wholesale set of edits, for very minor benefits, as far as I can tell. Also, having multiple smaller buffers does seem to be quite helpful when considering looping, which is something I didn't (and probably won't) implement. I was aiming for the smallest set of changes which allowed reliable multi-track playback, and retained the original feature set. I think I succeeded, but as the original "clients" (the MicroDexed project) couldn't get it to work, and couldn't provide a convincing example of it not working, I stopped where it is today.

@horstmaista, I realise I didn't respond to your post #95. No, playback from arbitrary positions isn't in there. I may come back to it at some point, as it shouldn't be too hard, but it probably won't be any time soon.
 
Thx for the quick response. For me it's working fine with 3 or 4 sample players. Haven't tested some features like crossfade, but pitch, forward loop are ok in my initial tests
 
@horstmaita, I've taken a look at starting playback from an arbitrary point in the sample file: you can find the first try on a new branch in my fork.

To use it, set up your playback objects as usual, but before you start playback, call something like playSdWav1.setPlayStart(play_start_arbitrary,22050);. This will cause playback to begin 22050 samples from the start of the file, i.e. 0.5 seconds in at the default sample rate of 44.1kHz. You still can't make a sudden jump in playback position while playing, though. Also, there's something wrong with stereo playback - I need to look into this further.
 
Back
Top