changing pitch of audio samples - TeensyVariablePlayback library

@horstmaita, I've taken a look at starting playback from an arbitrary point in the sample file: you can find the first try on a new branch in my fork.

To use it, set up your playback objects as usual, but before you start playback, call something like playSdWav1.setPlayStart(play_start_arbitrary,22050);. This will cause playback to begin 22050 samples from the start of the file, i.e. 0.5 seconds in at the default sample rate of 44.1kHz. You still can't make a sudden jump in playback position while playing, though. Also, there's something wrong with stereo playback - I need to look into this further.
Today I tested the AudioPlaySerialFlashResmp and the sound goes really bad when setting pitch up (only one semitone and has distortion/artefacts... ) which does not happen with the SD player. Also in the serial flash player, with this last release my teensy gets stuck if I try to play with the loop on, this does not happen with the previous one.
 
Today I tested the AudioPlaySerialFlashResmp and the sound goes really bad when setting pitch up (only one semitone and has distortion/artefacts... ) which does not happen with the SD player. Also in the serial flash player, with this last release my teensy gets stuck if I try to play with the loop on, this does not happen with the previous one.
You might want to try increasing the number of buffers and their size in ResamplingSerialFlashReader.h:
Code:
#define RESAMPLE_BUFFER_SAMPLE_SIZE 512
#define RESAMPLE_BUFFER_COUNT           7
This might help the "sound goes really bad", could help the loop issue too. I don't have serial flash fitted so can't test this anyway, so won't bug you about the Forum Rule to post a small complete program that compiles with the Arduino IDE and demonstrates the issue...
 
I see you made a lot of changes and improvements in this topic.
Could you point me to the best/most recent version of your various branches to try to adapt for my scenario, meaning playing polyphonic, pitched samples from SerialFlash ? (or even SD Card)
 
I've changed nothing since post #100, so https://github.com/h4yn0nnym0u5e/teensy-variable-playback/tree/feature/set-start-point is the most recent (October 2023). Best? Who knows? It seems to take months to get any feedback on changes, and even then it's never in the form of a sketch I can compile to reproduce any apparent issue...

I can see Nic's made a few changes, which appear at first glance to be unrelated, but enough to require some conflict resolution if one were to try to get to the point of submitting a pull request. As far as I know his repo doesn't have any of my adaptations to improve playback from a filing system.
 
I've changed nothing since post #100, so https://github.com/h4yn0nnym0u5e/teensy-variable-playback/tree/feature/set-start-point is the most recent (October 2023).
I can see Nic's made a few changes, which appear at first glance to be unrelated, but enough to require some conflict resolution if one were to try to get to the point of submitting a pull request. As far as I know his repo doesn't have any of my adaptations to improve playback from a filing system.
Thanks so much for your quick reply.
Hopefully i have some time next week to look deeper into it. It still surprises me that so few? users seem to have any interest in this topic, even it is the only way to produce pitched samples on the teensy - looks like the teensy is not that commonly used in this field or somehow really few people are interested in playing pitched samples at all with it.
 
live sequencer

LSEQ.png

 
It still surprises me that so few? users seem to have any interest in this topic, even it is the only way to produce pitched samples on the teensy - looks like the teensy is not that commonly used in this field or somehow really few people are interested in playing pitched samples at all with it.
I wouldn't say "only". Two other (I think) independent libraries with pitched sample playback on Teensy come to my long-term lurker mind:
Bleeplabs
Frank Boesing's library
...and I suspect there are others kicking around the forum that could be unearthed...as the new forum search works much better.

And I wouldn't say "so few users seem to have any interest..."
For this codebase, we have Paul's original non-pitched playback-->Sandro's rough pitch shift version-->Moo's cleaner, clearer version with examples-->h4yn0nym0us5e epic fixes and refinements. Threads at all these junctures (and a good many "how do I pitch shift?" posts) show recurring interest in the topic.

Activity comes in waves. People come and go. Maybe a version of the current library will end up in the official set someday? Frank's might have but for license and timing issues. Further refinements in sample playback would no doubt start an avalanche of new sound projects and eurorack modules.

<digression>If you can let go of the Teensy Audio Library and think in a perhaps less friendly / intuitive paradigm, there is a comprehensive library from pschatzmann that has pitch shift (3 different algorithms) and sample stretching. His "audio kit" examples target a hellscape of dev boards (pin revisions without changes in board revision numbers and vice-versa), but the generic "Arduino Audio Tools" library codebase should be applicable to our beloved Teensy with I2S audio shield.

Also: I've done quick and dirty pitch shifting in Circuit Python using ulab for linear interpretation. CP runs on Teensy 4.1, though it would be massive overkill for what I'm doing, but then again...not everyone thinks small. Circuit Python is far behind on the sound front but is actively developing synth and sample playback features. </digression>
 
Thanks so much for your quick reply.
Hopefully i have some time next week to look deeper into it. It still surprises me that so few? users seem to have any interest in this topic, even it is the only way to produce pitched samples on the teensy - looks like the teensy is not that commonly used in this field or somehow really few people are interested in playing pitched samples at all with it.
Sorry for a delayed response... I don't have any information on how this conclusion was arrived at about interest in this library.

I use the library.

I have used the library through breaking changes, which requires more than a little patience.

What I haven't done is be verbose about it on any forum, any topic. My interest in it may well have not been counted in the above statement.

I'm not qualified to speak on motivations people might have for publishing open source work, but I am willing to say that anything worth doing is worth doing correctly, and doing it regardless of opinions.

I have forked the library, will continue to watch the topic and the repository. If it appears to be abandoned, I'll attempt to maintain it.
At least in this way I can hope to preserve the results of efforts to know, navigate and use the code and perform updates on the device(s) built using it.
 
...
I use the library.
I have used the library through breaking changes, which requires more than a little patience.
...
I have forked the library, will continue to watch the topic and the repository. If it appears to be abandoned, I'll attempt to maintain it.
At least in this way I can hope to preserve the results of efforts to know, navigate and use the code and perform updates on the device(s) built using it.
Looking back through this thread, seems you managed 8-file playback before I made my changes! I don't know if you've tried those changes (see links in post #100 or #104), but I'd love to know if they're an improvement for your use case. I don't think they break anything, though I have extended the API a little when adding functionality.

I know Nic has made other modifications which would require some conflict resolution before he could merge my branch, but I've held off negotiating with him on this until I can make a good case for putting in the effort ... one element of which would be "look, other people are finding it useful!".
 
I did! It is noteworthy that I put that project aside waiting on improvements to the MTP responder code. I only recently brought it up to date and found that it works with the library as it gets pulled down by the library manager.

If your modifications are in a branch, I don't have that but will try it.
 
"Use case"

That got narrowed down from 8 files tested to 6 files for the sake of not pushing resources.
Then I came to my senses and realized that I would only ever use a single sample at a time.

Single sample, midi control, with a "stutter" feature that upon note-off event leaves the sample in a playing state but with a playback rate of 0.
The next note-on event pushes a new playback rate into the variable.

No sampling software I have does this, but I haven't tried everything out there. It's gimmicky but fun gimmicky. ( I do have Stutter Edit but that's not exactly a sampler. )

Here is that engine stripped down.

#include <MIDI.h> #include "SD.h" #include <Audio.h> #include <TeensyVariablePlayback.h> #include <USBHost_t36.h> MIDI_CREATE_INSTANCE(HardwareSerial, Serial1, MIDI1); USBHost myusb; USBHub hub1(myusb); USBHub hub2(myusb); USBHub hub3(myusb); USBHub hub4(myusb); MIDIDevice midi01(myusb); MIDIDevice midi02(myusb); MIDIDevice midi03(myusb); MIDIDevice midi04(myusb); AudioPlaySdResmp sample1; AudioOutputI2S2 i2s2; AudioFilterBiquad filter1; AudioFilterBiquad filter2; AudioEffectEnvelope envelope1; AudioEffectEnvelope envelope2; AudioEffectEnvelope * envelopeQueue[2] = {&envelope1, &envelope2}; AudioMixer4 outmix1; AudioMixer4 outmix2; AudioConnection voice1filter(sample1, 0, filter1, 0); AudioConnection voice2filter(sample1, 1, filter2, 0); AudioConnection voice1envelope(filter1, 0, envelope1, 0); AudioConnection voice2envelope(filter2, 0, envelope2, 0); AudioConnection patchCordVoice1A(envelope1, 0, outmix1, 0); AudioConnection patchCordVoice2A(envelope2, 0, outmix2, 0); AudioConnection patchCordOut1(outmix1, 0, i2s2, 0); AudioConnection patchCordOut2(outmix2, 0, i2s2, 1); double fwdMIDIPitches[97] = {.062500, .066192, .070112, .074324, .078745, .083425, .088390, .093645, .099212, .105112, .111360, .117986, //0 .125, .132425, .140305, .14865, .157495, .166855, .176775, .18729, .198425, .21022, .222725, .23597, //1 .25, .264865, .280615, .29730, .31498, .333705, .35355, .374575, .39685, .420445, .445445, .471935, //2 .5, .529725, .561225, .59460, .629955, .667415, .707105, .74915, .7937, .840895, .89090, .94387, //3 1, 1.05945, 1.12245, 1.18920, 1.259925, 1.334835, 1.41421, 1.49830, 1.58740, 1.68179, 1.781785, 1.88775, //4 2, 2.11892, 2.244915, 2.37841, 2.51984, 2.66967, 2.82841, 2.99661, 3.17480, 3.36358, 3.563955, 3.77549, //5 4, 4.423784, 4.48984, 4.75682, 5.03966, 5.33935, 5.65685, 5.99322, 6.34962, 6.72718, 7.12714, 7.55100, //6 8, 8.47570, 8.97970, 9.51366, 10.07936, 10.67868, 11.31370, 11.98650, 12.69920, 13.45440, 14.25442, 15.10200, //7 16 }; double revMIDIPitches[97] = { -.062500, -.066192, -.070112, -.074324, -.078745, -.083425, -.088390, -.093645, -.099212, -.105112, -.111360, -.117986, //0 -.125, -.132425, -.140305, -.14865, -.157495, -.166855, -.176775, -.18729, -.198425, -.21022, -.222725, -.23597, //1 -.25, -.264865, -.280615, -.29730, -.31498, -.333705, -.35355, -.374575, -.39685, -.420445, -.445445, -.471935, //2 -.5, -.529725, -.561225, -.59460, -.629955, -.667415, -.707105, -.74915, -.7937, -.840895, -.89090, -.94387, //3 -1, -1.05945, -1.12245, -1.18920, -1.259925, -1.334835, -1.41421, -1.49830, -1.58740, -1.68179, -1.781785, -1.88775, //4 -2, -2.11892, -2.244915, -2.37841, -2.51984, -2.66967, -2.82841, -2.99661, -3.17480, -3.36358, -3.563955, -3.77549, //5 -4, -4.423784, -4.48984, -4.75682, -5.03966, -5.33935, -5.65685, -5.99322, -6.34962, -6.72718, -7.12714, -7.55100, //6 -8, -8.47570, -8.97970, -9.51366, -10.07936, -10.67868, -11.31370, -11.98650, -12.69920, -13.45440, -14.25442, -15.10200, //7 -16 }; char* filename; char* fetchedFile = "TALK.WAV"; union { double pb1 = 1; uint8_t pb1uint8_ts[8]; } pb1; float voice1Avol = 0.8; float voice1Bvol = 0.8; float filter1Q = .7; int filter1Frequency = 16000; float filter2Q = .7; int filter2Frequency = 16000; long position1 = -999; long position2 = -999; elapsedMillis counter1; elapsedMillis counter2; boolean stutter1 = true; boolean sample1fwd = true; uint8_t sample1channel = 1; void setup() { MIDI1.begin(MIDI_CHANNEL_OMNI); MIDI1.turnThruOff(); MIDI1.setHandleNoteOn(sampleNoteOn); MIDI1.setHandleNoteOff(sampleNoteOff); midi01.setHandleNoteOn(sampleNoteOn); midi01.setHandleNoteOff(sampleNoteOff); midi02.setHandleNoteOn(sampleNoteOn); midi02.setHandleNoteOff(sampleNoteOff); midi03.setHandleNoteOn(sampleNoteOn); midi03.setHandleNoteOff(sampleNoteOff); midi04.setHandleNoteOn(sampleNoteOn); midi04.setHandleNoteOff(sampleNoteOff); SD.begin(BUILTIN_SDCARD); myusb.begin(); AudioMemory(96); sample1.enableInterpolation(true); sample1.setPlaybackRate(pb1.pb1); outmix1.gain(0, 0.9); outmix1.gain(1, 0); outmix1.gain(2, 0); outmix1.gain(3, 0); outmix2.gain(0, 0.9); outmix2.gain(1, 0); outmix2.gain(2, 0); outmix2.gain(3, 0); filter1.setLowpass(0, filter1Frequency, filter1Q); filter1.setLowpass(1, filter1Frequency, filter1Q); filter1.setLowpass(2, filter1Frequency, filter1Q); filter2.setLowpass(0, filter2Frequency, filter2Q); filter2.setLowpass(1, filter2Frequency, filter2Q); filter2.setLowpass(2, filter2Frequency, filter2Q); envelope1.sustain(1); envelope2.sustain(1); envelope1.release(1); envelope2.release(1); filename = "TALK.WAV"; } void loop() { myusb.Task(); midi01.read(); midi02.read(); midi03.read(); midi04.read(); MIDI1.read(); testfire(); } void testfire() { if (!sample1.isPlaying()) { envelope1.noteOff(); sample1.playWav(filename1); envelope1.noteOn(); } void sampleNoteOn(uint8_t channel, uint8_t note, uint8_t velocity) { if (channel == sample1channel) { if (sample1fwd == true) { pb1.pb1 = fwdMIDIPitches[note]; } if (sample1fwd == false) { pb1.pb1 = revMIDIPitches[note]; } sample1.setPlaybackRate(pb1.pb1); if (stutter1 == true) { if (!sample1.isPlaying()) { envelope1.noteOff(); sample1.playWav(filename); envelope1.noteOn(); } envelope1.noteOn(); } if (stutter1 == false) { if (!sample1.isPlaying()) { envelope1.noteOff(); sample1.playWav(filename); envelope1.noteOn(); } } } } void sampleNoteOff(uint8_t channel, uint8_t note, uint8_t velocity) { if (channel == sample1channel) { if (stutter1 == true) { pb1.pb1 = 0.0; sample1.setPlaybackRate(pb1.pb1); envelope1.noteOff(); } if(stutter1==false){ envelope1.noteOff(); if(!envelope1.isActive()){ sample1.stop(); } } } } void audiodebug() { AudioProcessorUsageMaxReset(); Serial.print(AudioProcessorUsageMax()); Serial.print(" , "); Serial.println(AudioMemoryUsageMax()); }
 
Doesn’t look as if anything I’ve done will help much with that, though I think I got the looping to work properly, so you wouldn’t need your testfire() function to restart the sample when playback stops. Or maybe I’m misreading the code…
 
That's a holdover from initially testing hardware, it used to fire off the 8 samples at different rates.
another holdover would be the audio memory set that high.

I have noticed there is more interesting functionality that I haven't yet tried.
 
OOf, I made a real mess out of that sketch stripping it down, leaving UI variables in, leaving the testfire function in - and cutting off the last brace when axing the function below it... etc

I compiled the sketch that mess came from using the alternative branch, no problems, no difference in operation.

I certainly encourage any effort to reconcile the versions. Thanks to both of you for your work.
 
For whatever it's worth, here's the build. Using Adafruit 'seesaw' encoder board, UDA1334 codec, Triad TY-250 transformers ( 20hz-20khz , 2.8db insertion loss )
 

Attachments

  • IMG_2921.jpeg
    IMG_2921.jpeg
    285.4 KB · Views: 58
  • IMG_2922.jpeg
    IMG_2922.jpeg
    289.1 KB · Views: 63
  • IMG_2923.jpeg
    IMG_2923.jpeg
    296 KB · Views: 62
  • IMG_2924.jpeg
    IMG_2924.jpeg
    394.7 KB · Views: 64
I have the "alternate" library compiling with zero warnings. My choice was arbitrary. I made the most sane changes my skills allow. Nothing significant, just silencing annoying warnings. Put the device to a torture test for a while, no undesired behavior, even with a fast arpeggio.

You can see these changes on my github *branch*. (KreoPensas)

Warnings may only be warnings, but devices that act possessed are absolutely no fun. Nobody wants those.

 
Long time lurker, first time poster. I want to begin by thanking everyone for the hard work that went into this variable playback library. There are MANY people that benefit from this library whether they post here or not.

I've experienced issues with audio playback essentially locking up the teensy for 3-4 seconds before continuing playback normally. This moment of lockup / freezing happens seemingly randomly once audio playback has begun (it's usually about 10 seconds into playback but it depends). During the freeze, the audio will sound distorted and slowed / glitchy beyond recognition while also freezing up i2c communication with a small OLED screen (the screen is on i2c port 2 in an attempt to avoid conflict between the audio shield and the screen.)

I've experienced this freezing phenomenon both when I was attempting to play a full song and when I was attempting to sequence short samples at different pitches. The freeze seems to happen at least once every time I run the program and has happened across multiple projects using both teensy 4.0 and 4.1.

I'll attempt to provide sample code when I get free time in the coming days so that this issue can be repeated. Any advice would be appreciated.

Thanks for all of the hard work on this library!
-Ben
 
Long time lurker, first time poster. I want to begin by thanking everyone for the hard work that went into this variable playback library. There are MANY people that benefit from this library whether they post here or not.

I've experienced issues with audio playback essentially locking up the teensy for 3-4 seconds before continuing playback normally. This moment of lockup / freezing happens seemingly randomly once audio playback has begun (it's usually about 10 seconds into playback but it depends). During the freeze, the audio will sound distorted and slowed / glitchy beyond recognition while also freezing up i2c communication with a small OLED screen (the screen is on i2c port 2 in an attempt to avoid conflict between the audio shield and the screen.)

I've experienced this freezing phenomenon both when I was attempting to play a full song and when I was attempting to sequence short samples at different pitches. The freeze seems to happen at least once every time I run the program and has happened across multiple projects using both teensy 4.0 and 4.1.

I'll attempt to provide sample code when I get free time in the coming days so that this issue can be repeated. Any advice would be appreciated.

Thanks for all of the hard work on this library!
-Ben
just a blind guess, are you streaming the samples from the SD Card or from something else ?
 
just a blind guess, are you streaming the samples from the SD Card or from something else ?
I'm streaming samples over SD card, sorry for not making that clear. When using the teensy 4.1, I tried both available SD card slots with the same results. I'm using a 32 GB Kingston canvas select plus.
 
I'm streaming samples over SD card, sorry for not making that clear. When using the teensy 4.1, I tried both available SD card slots with the same results. I'm using a 32 GB Kingston canvas select plus.
Using the built-in SD card slot will work much better than one on the audio adaptor - it maxes out at over 20MB/s for single file reads, but you should probably expect ~10MB/s if playing multiple samples. That's enough to stream 16 mono files comfortably.

The "vanilla" streaming code written by Paul for the Teensyduino library, and Nic for the variable playback library that this thread is about, relies on doing SD card access within the audio interrupt. This is quite problematic for all sorts of reasons, hence my modifications linked from post #104 etc. These move the SD access to essentially make it part of your loop() function - think of it as being invisibly tacked on at the end, without your having to think about it. You can therefore open/read/write/close SD files as part of your sketch without causing mayhem, and the audio interrupt doesn't barf if SD access takes more than 2.9ms. The downside is that if your loop() code takes 20ms to run because you're using a slow I²C screen, your audio objects have to have at least that much audio buffered for every channel that's playing, which takes extra memory.

My modified code doesn't really allocate enough buffer memory, but it's fairly straightforward to edit the relevant files according to your use case - look at the top of ResamplingSdReader.h and you'll see:
C++:
// Settings for SD card buffering
#undef RESAMPLE_BUFFER_SAMPLE_SIZE
#undef RESAMPLE_BUFFER_COUNT
#define RESAMPLE_BUFFER_SAMPLE_SIZE 512
#define RESAMPLE_BUFFER_COUNT         7
This gives 7 buffers of 512 samples, so 3584 samples or 58ms buffer for a mono file. That's possibly a bit marginal if the SD card suddenly decides to take 100ms to do a read, as can happen. And if your file is stereo, you should double the buffer!

If you have PSRAM fitted, you can use setBufferInPSRAM(true) to allocate the buffers in your PSRAM, which allows you to have more of them, or bigger ones - a buffer size of 2048 samples is close to optimal for a decent SD card, but you should play around to see what works for you.
 
Sorry if i asked this previously @h4yn0nnym0u5e, i made a new attempt to use your modifications by dropping out temporarily other CPU and RAM hungry code to see how far i can get. I noticed that some short percussion samples end with a (full volume) click while others do not. This behaviour is consistent, it is always the same set of samples making a click while the others have no issue. All samples are faded out to zero and strangely this only happens when playing from SD Card. From Progmem (with same data) it makes no clicks.

I played a lot with the buffer values but that does not seem to make any difference in this regard. I still can't really use your wonderful work (most probably of other issues) but i am wondering if this is somehow known to you or others.



 
Don't think this has been observed before. If you can post a simple sketch I can drop into the Arduino IDE, together with one of the troublesome samples, I'll try to find time to take a look.
 
As much as i would like to do that i don't think i will be able to provide it and probably is an issue coming from other parts of the code.
I was hoping it is kind of a known issue because of some changed code in variablePlayback.
Please ignore my issue as long as it is not reported by others with a more manageable code base.
 
Back
Top