changing pitch of audio samples - TeensyVariablePlayback library

positionhigh said:
I understand you do not have much free time but just to let you know - you can test/experience our project with just a bare teensy 4.1 and your PC connected via the onboard MicroUSB and a Webbrowser.
You even will hear audio over USB in the Webbrowser, see the "screen" and can control everything by PC keyboard.
Let me know if you like to get any help or assistance.
Click to expand...

I've taken a quick look, but it's probably not a good use of my time to try to get my head round your project in order to use it as a debugging tool. Also, I might get distracted playing with it!

If you can figure out a minimal Arduino sketch which demonstrates the 10+ seconds "with glitches", I can take a look. I use the 1.8.19 IDE at the moment, because it seems to be more robust than the 2.x and "everyone has it"; running on Windows 10. If you think it's useful to post a WAV file (probably needs to be ZIPped in order to attach it), that's awesome, but I can easily put something together here if not. My current test file is a frequency sweep, easy to tell if it's pitch-shifted or playing backwards.
 
Last edited:
Moo said:
Hi h4yn0nnym0u5e,

Sorry for a long period of silence from me.

Reading the SD card during an interrupt is less than ideal so your async method could avoid that, and thats a great thing.

I would love to, but I don't have much time to write 'recreational' code, at the moment. I'm coding up to 40 hours a week at work usually. I do find some time to code, sometimes an hour after work, but I have to prioritise what I do with that time. so I try to find the intersection of the most useful/necessary thing and the thing I enjoy doing most. (and I tend to go down rabbit holes quite often...). And occasionally I find time to binge 'recreational' code.

At the moment, 'recreationally', I'm mostly working on my polyphony/sampler/sequencer. Im having a lot of fun writing it and many cool elements are all coming together. It is a modular music machine in teensy from the ground up , decoupling sequencers(step-sequencers, midi files/loops) from voices(sine,wav,raw,wavetable,...)....

The teensy-variable-playback library needs so many improvements at the moment - I don't think its even working at the moment with SD! as PositionHigh said - he needed to manually insert disable and enable irq methods. Theres so much stuff that needs attention/improvement/efficiency/performance, and I am planning on spending some time soon on addressing that

So if you can make it stable with no pitch changing, ...at some point in the future, when I've improved my variable-playback, I will try to find time to apply pitch changing to your code if I can...

let us know how far you've progressed on the Async code. Its very interesting, I've tried to think how I could apply this concept to my library and I havent figured it out... yet.... :_)
Click to expand...

Hi Moo

No worries about no response, though it's great to hear from you. If you keep lurking and chip in with the occasional "please don't do that!", all will be fine.

I'm actually starting from a fork of your code, and making what I hope are minimal changes to cause file reads to happen inside yield() using Paul's EventResponder class. This means they're effectively part of the user sketch, not inside the audio update, and shouldn't clash with other file I/O. Although I'm only testing with SD files, I have hopes that few / no changes will be needed to make it work with LittleFS, SerialFlash etc. The only issue I've had so far is with array playback, because it isn't a filesystem - I had to learn about template specialization there... so all your pitch changing and looping stuff is entirely untouched and still appears to be as good as ever.

All development is currently on this branch, if you want / have time to take a look.
 
h4yn0nnym0u5e said:
I've taken a quick look, but it's probably not a good use of my time to try to get my head round your project in order to use it as a debugging tool. Also, I might get distracted playing with it!

If you can figure out a minimal Arduino sketch which demonstrates the 10+ seconds "with glitches", I can take a look. I use the 1.8.19 IDE at the moment, because it seems to be more robust than the 2.x and "everyone has it"; running on Windows 10. If you think it's useful to post a WAV file (probably needs to be ZIPped in order to attach it), that's awesome, but I can easily put something together here if not. My current test file is a frequency sweep, easy to tell if it's pitch-shifted or playing backwards.
Click to expand...

That will be complicated but i understand your point. That was the reason why i tried to drag you in to see actualy the same issues we see :_) Since your help is priceless i wil try to find a method to show it otherwise but my fear is that is not really a problem in your code standalone but putting it together with all the other 100 things.
 
Hey, moo -
very happy to hear some news from you after a while about your library. You know i am more on a novice level but when you would ask me, i would drop the littlefs and extra stuff for flash. The default serialflash library does not have all the extra fancy stuff like subdirectories, but from my experience it is less cluttered and much responsive, at least for sample playback. So in my opinion, one abstraction layer could be removed, like it was in the previous versions of variableplayback.
To get it to work with SD Card and multiple files is a complete other dimension - I hope that h4yn0nnym0u5e will come up for that with some more miracle work.
 
Hey h4yn0nnym0u5e,
i am not sure how fast i can come with an demo code.
I don't think it is that much helpful but hope to get you more involved somehow:_) first half of the video is Nics Library playing from flash. Second half is with your current code state for SD card.
Dont't turn volume up to much since there will be nasty artefacts. No question, it is not better with the default of Nics Library, however the problem might to be our own issues, not the library - im kind of clueluss where to look for the problem, first.

https://youtu.be/yz_a2vfgIeM
 
positionhigh said:
Hey h4yn0nnym0u5e,
i am not sure how fast i can come with an demo code.
I don't think it is that much helpful but hope to get you more involved somehow:_) first half of the video is Nics Library playing from flash. Second half is with your current code state for SD card.
Dont't turn volume up to much since there will be nasty artefacts. No question, it is not better with the default of Nics Library, however the problem might to be our own issues, not the library - im kind of clueluss where to look for the problem, first.

https://youtu.be/yz_a2vfgIeM
Click to expand...
You don't have to be fast, I think there are quite a few things I need to do before it's worth doing a proper test with real-life applications!

One thing you could perhaps try fairly easily is to change the buffer sizes and count. In src/ResamplingSdReader.h you'll find these lines:
Code: 
#define RESAMPLE_BUFFER_SAMPLE_SIZE 512
#define RESAMPLE_BUFFER_COUNT 	7
RESAMPLE_BUFFER_SAMPLE_SIZE is in samples, so this says a buffer is 512 samples long (1024 bytes), hence it'll last for 4 audio updates assuming a mono WAV file - it'll only last for 2 updates if you're using stereo. It will then be reloaded in the foreground / sketch / EventResponder code, and given there are RESAMPLE_BUFFER_COUNT = 7 buffers, that could take up to (7-1)*4 = 24 updates, or about 96ms, before the playback is starved of audio data. That's fine for a single object and a decent SD card, but obviously gets more marginal as the number of playback objects is increased. Also, if your code doesn't yield() quite often, that prevents EventResponder from doing the pending reloads. And, big reads are more efficient than little ones, so using 1024-byte buffers is a bit on the small side.

So, try increasing RESAMPLE_BUFFER_SAMPLE_SIZE from 512 to 2048 or more (must be a power of 2), and possibly increase RESAMPLE_BUFFER_COUNT as well. You should also have a good hard look at your loop(), and instrument it to see how often it exits, assuming you don't already know. yield() is called on loop() exit, and also on delay() calls, but using the latter (except on super-rare occasions) should be an absolute no-no in a real-time application like this. If your loop() has multiple time-consuming operations, either use a state machine to run each one in turn on separate loop() iterations, or put a yield() call between the long operations. "Long" is anything over, say, a quarter of the total buffer duration divided by the number of playback objects. So if you have 8 objects playing, each with a 240ms buffer, operations can ideally take no more than 7.5ms before yield() needs to be called again.

For testing, please keep it super simple, at least to start with. It's lovely to have a nice tune, but it's pretty much hopeless for early debugging! Can you play a single sample at 1x? Two samples at once? Four? Different playback speeds? Does it help to turn the display or web page off? When it breaks, record that to a WAV file, along with the same thing from flash (presumably not broken), and zip and attach it here. Boring sine waves are excellent samples, or perhaps a sweep, so it's super-easy to track where samples have gone missing or got duplicated.
 
h4yn0nnym0u5e said:
You don't have to be fast, I think there are quite a few things I need to do before it's worth doing a proper test with real-life applications!
One thing you could perhaps try fairly easily is to change the buffer sizes and count. In src/ResamplingSdReader.h you'll find these lines:
Click to expand...

Very helpful tips, thank you. As one of the first steps i will try to strip out everything possible and observe if any step of it makes an difference. Since everything is interwoven, that might take a moment :_)
will get back to you with what i find.
 
i was thinking to revert all the bells and whistles with variable playback, like interpolation and looping and then there won’t be as much noise. also you can set this up to debug on your local mac or linux if you are that way inclined. let me know if you wish to go down the rabbit hole :)
 
Moo said:
i was thinking to revert all the bells and whistles with variable playback, like interpolation and looping and then there won’t be as much noise. also you can set this up to debug on your local mac or linux if you are that way inclined. let me know if you wish to go down the rabbit hole :)
Click to expand...

Nah, don't do that, I'm having too much fun! Besides, some of that is independent of file reading (e.g. interpolation), and optimising looping is next up on my hit list anyway. There will no doubt be a few gotchas related to changing loop points while playing...

I was thinking ... the way the code works now, playback speed could fairly readily be an audio input and modulated by e.g. an LFO. Now there would be fun.
 
Couple of updates today. playbackRate < 0.0 should now work a bit better in terms of predictive preload, and it's possible to use PSRAM for the buffer memory so you aren't risking running out of heap when using many / big buffers. To use PSRAM, do something like playSdWav1.setBufferInPSRAM(true); Or, of course, playSdWav1.setBufferInPSRAM(false); if you want to go back to the normal buffer allocation in the heap.
 
Quick update, now does a better job of reloading buffers if yield() hasn't been called for a while. Also outputs a zero sample if the required one isn't in a buffer, rather than attempting to read from the filesystem during the audio update interrupt. This should improve stability considerably, especially if the filesystem is also being used for non-audio access.
 
h4yn0nnym0u5e said:
Quick update, now does a better job of reloading buffers if yield() hasn't been called for a while. Also outputs a zero sample if the required one isn't in a buffer, rather than attempting to read from the filesystem during the audio update interrupt. This should improve stability considerably, especially if the filesystem is also being used for non-audio access.
Click to expand...

I can confirm this helped a lot in certain circumstances. Instead of crashing or very nasty audio artefacts, it is more solid now with less agressive audible issues. I will provide a audio demo shortly.
 
i did however not have so much sucess with the setBufferInPSRAM(true), i tried to call it before the ->playWav, after it and both but the results were equal or worse than without it.
Maybe something in the main application needs to be done to make it work (better) than without ?
The PSRAM "test" works out fine and also when using it as space for longer delay/reverb times.
 
Glad it’s working better for you.

Using PSRAM lets you use much larger buffers, which ought to help if your application blocks or doesn’t yield() for long periods. It is slower than heap RAM, so if timing is super-critical it may make matters worse. You must enable it before using playSdWav.play(), afterwards has no effect, or possibly only takes effect on the next .play() call. PC off for tonight so can’t check!
 
Another update pushed, giving further stability improvements. I'm still seeing the occasional temporary audio "stall", which may be a bug (more likely) or possibly an SD card artefact.

My test code:
  • has 7 audio buffers set to 2k samples each (14k samples, 7x 46ms = 325ms total buffer)
  • has two playback objects, connected to left and right outputs
  • repeatedly plays a one-second file through both objects at the same time
  • varies the playback direction randomly
  • varies the speed from 0.5x to 2.0x
  • has an artificial sketch stall, so yield() is only called every 40ms
  • also has a text file open and prints a bit more of it to the serial monitor every so often
It's also a total mess, so i'm not posting it!

As part of this debug cycle, I looked at the speed impact of using PSRAM for buffering. As far as I can see there's no impact, presumably because the timings are dominated by SD card read and interpolation calculation timings.
 
That are great news!
I tried several times with much stripped out code of our project but all in all, so far no real difference. My feeling is that the SD Card loading is not getting enough/long enough cycles to get the data in. And so, the buffering to PSRAM can't really help.
One question : I still have to add the __disable_irq() and __enable_irq() around the file seek + read in IndexableFile.h to have it running stable. Wondering why that is, since you guys seem to have no issues, without that?

Code: 
__disable_irq();
            _file.seek(seekPos);
            int16_t bytesRead = _file.read(next->buffer, BUFFER_SIZE * element_size);
            #ifndef TEENSYDUINO
            if (!_file.available()){  
                _file.close();
                _file = open(_filename);
            }
            #endif
 __enable_irq();
 
Hmmm ... all that strongly suggests you may have somewhere in your code where the SD card is being accessed under interrupt. That's been a major cause of instability for me while doing these SD playback / recording objects, and it's not always easy to see where the offending code is. Even closing a file is enough to cause a crash. You definitely shouldn't need to disable interrupts in loadBuffer() that way, all file seeks and reads are now happening in the foreground code and the worst that should happen if interrupted is that they take a little longer.
 
I've downloaded your MicroDexed-touch code for a brief look ... not to make any changes!

As it's fairly complex it's really hard to see where you might be stripping stuff out to test my updates, and you appear not to be in the habit of doing stuff like that on a branch and pushing it up for all to see. Indeed, despite recent releases, I can't even see the source file changes corresponding to those releases. Very hard to follow, so I'm not going to try further.

One file I did look at in some detail was dexed_sd.cpp, as it looked as if it might be relevant. One immediate thing I noticed is it's full of Audio[No]Interrupts() calls, presumably (or at least, we hope to be soon) historical. Rather worse, you're in what I think of as the nasty habit of dealing with errors by an early return from a function - get_sd_data() is an example. Why is it nasty? Because you leave audio interrupts off as a result...

On another note, I've not tested looping in the latest pushes, and I rather suspect it won't work, so if your stripped-down code still uses looped playback that might be a cause of remaining issues. Otherwise, the only thing I can encourage is to use large buffers in PSRAM for initial tests.

Oh, and if you have access to an oscilloscope, use it to instrument the code in real time, e.g. the audio updates, SD card reads, your loop(), display update code and so on. Good functions for SD read instrumentation are in libraries/SdFat/src/SdCard/SdioTeensy.cpp, either SdioCard::readSector() or SdioCard::readData(). And yes, even Bill Greiman uses early returns. I still despise them.
 
Yes, i need to work with git more regulary, i am still kind unfamilar with it and can use only the most basic functionality. So until getting to the next breakthrough we will concentrate on the code and our users prefer binaries anyway.

Thanks again for all the tipps. I tested without any loops, just short one-shot samples.
It seems not to matter much, if i fire 1, 2 ... 5 samples at the same time.
I like the timing idea with an scope. I will see if i can do that (only have a very lowcost-pocket-scope)


Thank you for taking some time for looking into MicroDexed-touch and dexed_sd.cpp and sure there are problems. However, i am 99,9% certain that nothing in there affects the audio playback. All the file operations there are only happening before playback starts.
Nothing on the SD Card should happen, while playing, except loading the samples of course.
 
I'm close to getting looping working OK, but have run out of time for the moment to get a reasonably clean update pushed to GitHub. However, I've just discovered that EventResponder doesn't do quite what I thought it does, as in if you have multiple samples playing and creating events to request filesystem reads, only one event gets processed per yield() call. I thought every pending event got processed... A proper fix requires changes to EventResponder in the core code - we shall see if that gets any approval - but in the mean time a bodge is to put something like this in your loop():
Code: 
void loop(void)
{
    // your usual code here - make it as quick as you can!


     // force extra event responses per loop()
    for (int i=0;i<10;i++)
      yield();

}
You should probably stick in as many iterations as you think you'll have samples playing. I think yield() is cheap if there's nothing to do, but this is most definitely not production-quality code! But it might get you going.
 
Looping now working OK, as far as I can tell, and various pieces of debug code are commented or #ifdef'd out. You will still need to put some extra yield() calls into your loop, until the correct approach for EventResponder is determined. In the above example they're all in one place, but you could of course scatter them throughout your loop() to even out the CPU load.
 
Yes, there is definitly a difference when putting in more yield().
If i disable all or most of the other sound generation beside the samples, i can get 3-4 samples playing nearly without glitches.
I retried the PSRAM option but not seen any difference with or without it (maybe even a bit worse with it being on)
In your latest code with the Loop-improvements, i had to enable the disabled #define ABS line to get it compiling without strange errors for several other places around.
 
As a side topic - i think this is an issue with the audio library in general with wav files,
when stopping samples, so they are not playing to the true end, there are always clicks or glitches.
I tried to fix it with the provided envelopes and that kind of makes it fixable, but that also messes up some or all the sample starts (skipping, artefacts at sample start etc.)
Not sure if you can do anything about that because, as said, this seems to be a general issue with the audio library.
 
I need to do some more testing with multiple files, for sure. If I find and fix something, all well and good, I can let you know and you can test again.

If not, I really am going to need more quantifiable reports than "there is definitely a difference", "3-4 samples playing nearly without glitches" and "not seen any difference with or without it (maybe even a bit worse with it being on)". Small example sketches I can compile and run using Arduino 1.8.19; recordings of output, ideally via USB for sample-accurate investigation, possibly with a reference of "this is how it should be"; maybe images of scope traces, if recordings aren't working. This stuff is phenomenally hard to debug with the available tools, and my limited time will be way more productive if I can start from some reproducible test cases.

Yes, I'm not sure what's going on with the abs() thing - if the bodge I put in and commented out needs to remain in place for now then so be it, we can fix it later in the tidying-up phase.

The audio library is fairly basic when it comes to starting and stopping pretty much anything, be it sample files or synthesized waveforms. If you tell it to stop, it will do so, sometime in the next 2.9ms, and regardless of where the source has "got to in the waveform". It's 100% up to the application developer to deal with this. In my SDpiano example I made each voice a class derived from AudioStream, with two built-in envelopes (for stereo), both memory playback (for low latency note starts) and filesystem playback (for long samples), and mixers for velocity sensitivity and mixing the memory and filesystem players. Its implementation of stop() then triggered the envelope release, and its update() stopped playback only when the envelope went inactive. The envelopes are only really used for their release phase, so have zero delay, attack, hold and decay, and a sustain level of 1. I haven't noticed any messing up of sample starts, though of course that could be related to initial playback being from memory. Yes, it's fairly complex. No, you don't have any option but to do it properly and then test it thoroughly. It took me quite some time to get it right.
 
You must log in or register to reply here.
Back
Top