changing pitch of audio samples - TeensyVariablePlayback library

positionhigh said:
As much as i would like to do that i don't think i will be able to provide it and probably is an issue coming from other parts of the code.
I was hoping it is kind of a known issue because of some changed code in variablePlayback.
Please ignore my issue as long as it is not reported by others with a more manageable code base.
Click to expand...
I don't see why you can't provide an example. Even if you don't currently have the Arduino IDE installed, it's simple enough to do, and probably more productive than "[playing] a lot with buffer values". It is also the canonical method for this forum of providing a worked example for a code maintainer to start from - otherwise they're just thrashing about in the dark. If you can produce a demonstration sketch, then the fault lies either in that (which should be apparent for a simple sketch), or in the library, in which case it can be fixed for the benefit of all users. If you can't then as you say the problem lies in your code, which only you can fix.

To be fair, sometimes a bunch of words can at least trigger a "try this" response, as in post #119 above. But if "this" doesn't work, the next request will always be "show me a small example". If that's not forthcoming, chances are high that even if there is a bug, it will not be addressed until someone can be bothered to make the effort to create the example.
 
Sorry for resurrecting this thread, but I just wanted to describe how I'm managing samples for my groovebox so far:

I have the Teensy 4.1 fitted with 16MB of PSRAM, and I've been using Nic's teensy-variable-playback library in conjunction with his other teensy-sample-flashloader library to load samples into PSRAM from the SD card. Let me explain briefly how it works.

While a sequenced pattern on my groovebox is playing, selecting another pattern to jump to causes the next pattern to be queued up until the remaining pattern bar finishes. Nic wrote the code in my repo to asynchronously load the next pattern's samples when the next pattern is first queued up. So it finds the next pattern's sample names and starts loading them onto the non-active 8MB heap of PSRAM. Once the pattern change happens, the active/inactive heaps are "swapped" in the sense that the code to play the audio files points now to the memory heap location in PSRAM which is now marked as "active". So effectively, 8MB of PSRAM is dedicated just for swapping in/out samples between patterns.

This to me seemed like a decent compromise, and seeing as how my groovebox has 16 banks of 16 patterns (with 16 tracks and 16 sample voices available) per project, that's a lot of potential samples per project. However, the downside is that there is still the limit of 8MB worth of samples being playable at any one time. Also, I've only tried the asynchronous loading approach with short one-shot drum samples. For larger samples, this approach sort of breaks down since you want any samples to immediately play when the next pattern starts, and loading large samples might not happen in time.

So, I would love to use this SD-streaming approach to get around these sorts of constraints if possible, similar to how the Dirtywave M8 handles sample streaming.

But having some of the variable playback features like pitch changing and ideally the looping would be lovely too.

Has there been any updates in this domain? Or are there any new libraries or forks of the Audio code which achieves this since this thread was last updated? I would also be willing to try to implement this stuff on my groovebox in order to try and see if I can help out moving the ball forward here.
 
We came to a very similar conclusion. Also switched to psram with a slightly modified sample-flashloader.
This solved most of the known issues.
Currently working on a sample/editor/slicer.
SD-streaming would be wonderful but i lack the skill to get this working anything near the m8.
 
@graydetroit, you could give my changes a try. Please see posts #104 and #122 for links to my branch, and ways of tuning that for (possibly) better results - I wasn't very generous with my default buffer allocations.

In order to "cue up" a sample for playing, you could maybe try by setting the playback rate to 0.0, start playing using play(filename), and then set the playback rate when you're ready to trigger a sample - it should have been pre-loaded (if you gave it enough time), and thus start within one audio update period, i.e. 2.9ms. I think this will work, but I haven't tried it. It will probably wait outputting the first sample continuously, so if that's a zero so much the better! If not, you could put a mixer or amp in the signal path with its gain set to 0.0, and change the gain when you start playback. With 16 tracks you may find it easier to have 32 playback objects in pairs, one playing and the other being cued up ready for the next step. Even if you give each one 32kB of buffer (by editing the library), that's still only 1MB of your PSRAM used; bigger buffers mean more efficient filesystem reads.

If you have an issue, please post a small sketch that reproduces it, which I can drop into Arduino IDE, and run.
 
@h4yn0nnym0u5e I think there's a couple things giving me pause after thinking about it. Sure, I'm sacrificing 16MB of PSRAM primarily for samples / sample swap, which seems kinda sad, but the benefits of having instant playback with Nic's libraries seems pretty critical for my use case oriented towards one-shot samples in a drum machine context. I think the buffered approach could in theory work, but I think the caveats and tradeoffs and strict timing concerns I have are pushing me away from it.

I think what I may end up doing is eventually trying to incorporate the buffered-SD stuff for enabling a polyphonic sample synth engine. I was just discussing this the other day, the fact that there's a dearth of modern polyphonic sampler synths, whereas they were more common in the 80s-90s. I think this use case better suits the buffer architecture for me since users will want to take, say, a ~10s sample of a single string or pad note from a synth, and use it to play it across several notes/pitches in order to emulate playing a real synth. So here, there's no desire to jump around the sample / abruptly change the play-start position.

There is definitely a different use case for taking a very long sample and doing slicing, but I'm not really interested in featuring slicing on my groovebox. I'm aiming more towards a fast-paced, nimble, simple workflow overall. It just so happens to coincide with the fact that doing much more is just way out of my depth as a programmer 😅
 
What I suggested in my post should give you “instant” playback, or at least as instant as the audio library is capable of - there’s always the 2.9 millisecond uncertainty. There will be a few milliseconds’ wait when you call play(), but after that the first few kB will be in memory and ready to trigger.

Surely the best way to deal with your concerns is to try a simple example, entirely separate from your existing code base. That will give you (and the rest of us, assuming you post the results) an idea of whether my suggestion has any merit at all. If it works, great; if it’s marginal, maybe I can make improvements; if it’s hopeless there’s probably a bug I should fix!

Obviously there would be significant work for you going from a completely real-time approach of “it’s this millisecond so trigger these samples“ to a more predictive one of “the next sample to play will be X, so get it ready”. I’d never advocate making a big leap like that in one go :) Look at my development of this library branch. I just checked, and I have 8 branches and made 30 commits to get to the current state. In many ways that’s mirrored exactly what you want to do, going from load on demand to load what will be demanded.

It occurs to me that if you are doing a lot of sample cueing, e.g. all 16 tracks at very short intervals, the pre-buffering time may cause issues. In that case having some samples in RAM and others on SD might help, but it makes things even more complex. Definitely not worth considering unless desperate!
 
Thought I recognised your name … we had a dialogue about starting buffered SD playback from an interrupt handler. In the end I haven’t rolled that in, as it was going to have massive ramifications for a fairly unusual use-case … sorry. The same issues are very likely to apply here, so any test code you do to try SD playback should start out by not triggering from interrupt.

I’ve got a glimmering of an idea on how to duplicate my AudioPreload capability for this library. If you write and report back on results for a simple test of 16-track SD triggering, that may give me the motivation to implement preload, which may in turn remove the issue with starting from interrupt.
 
Alright, I'll give it a shot with a small Arduino sketch prototype, I've been needing to solder together another Teensy anyway.

Yes, any direction I end up going in with the sample playback, I will want to be using that clock library which uses interrupts, so I hope that doesn't discourage you too much. But that can come later.

For now, I will try to make a basic sketch which alternates back and forth between two sequenced patterns using 32 total sample playback objects (16 tracks x 2 sample objects). 8 tracks will be focused on short one-shot drum samples, while the other 8 tracks will be dedicated toward longer samples for synthesizer emulation.

One thing I should mention is that a feature I currently have on my groovebox is the ability to assign 2 sample files to a single playback object, so that a single track can trigger two different samples (not at the same time though, since the samples share a single playback object, of course). Call this the dual-sample-single-slot feature, for short.

I really like this dual-sample-single-slot feature, and I'm wondering if the 32 (16 x 2) playback object approach using the buffered audio stuff can do double duty as enabling this same feature, as well as for cueing up the buffers for the next pattern before the pattern switches. But this seems more convoluted, having to manage which sample is currently playing and finding the free sample playback object to use when the next pattern starts. Not impossible, just more complex. Can't I just reuse a single playback object for the feature I'm describing above, like will there really be a noticeable latency while the buffer reloads? 2.9ms seems like it should be basically imperceptible, but for a drum machine, maybe not.

Ok, I'll get back to you with a sketch hopefully soon.
 
I think it’s typically about 5ms from calling play() to the sample being ready to play on the next audio update. That’s not too bad for only a few files, but worst-case with triggering all 16 tracks at once it’d be 80ms, which you’d definitely hear!

I’m not sure if the library needs to “understand” your dual-sample-single-slot (or DS3, for actually short? :) ) feature. Fundamentally a playback object is either playing or not; the “cued” state is sort of playing, albeit in a non-sound-emitting way. When you start playing you can choose any file or memory block you want, but once stopped there’s no record of what was played last - it’s your architecture that imposes the concept of only having two samples available to the object at any one time.

I’m starting from the premise that you have 16 tracks (aka slots? Or is that a different thing?), each of which currently has its own private playback object. When trigger time rolls round, you pick one of the two samples and fire it.

For better SD triggering I’m proposing adding another playback object per track. While running, you’d have one (A) playing (or idle because the sample has played in full), and one (B) cued up ready to play. When trigger time rolls round, you fire (B) the cued one, then look ahead to when the next trigger will be for that track, and cue up (A).

I think you’re safe to fire a cued track even from interrupt, provided your cueing up process has been done in non-interrupt code. Firing consists of setting the playback speed to non-zero, and signalling the foreground code to cue the other object up.

I‘ve glossed over the “look ahead” bit, as that’s possibly quite complicated? Depends on your data structures… if you’re performing I guess you may end up having to discard cued-up objects and re-cue them.
 
@h4yn0nnym0u5e here's a sketch I made. Don't want to post it inline here in the forum since the formatting is not super great. Here is a google drive link to the samples I'm using if you want to run it.

I haven't messed around with any pitch variation or anything, but surprisingly I'm able to sequence 12 tracks without crashing. The
RESAMPLE_BUFFER_SAMPLE_SIZE buffer size I'm using in the ResamplingSdReader.h file is 2048.

However, 8 of the tracks are short ~100kb or less one-shot samples. The last 8 tracks are dedicated to playing a ~1s long synth sample. And when I try sequencing 8 of these tracks playing the same sample consecutively, I seem to get crashes. So I left 4 of those tracks unsequenced (commented out in the sketch with a note).

Also, I wrote some stuff to identify which sample object is active for the current track so that I can determine which sample object to use right before the next pattern starts. See the setSlotsForNextPattern() function. I commented most of it out because it seems kinda buggy.

I intermittently hear some pausing on the overall timing, like where it pauses on a step a bit. When trying to use PSRAM for the buffers, the sloppiness is way more apparent across various tracks even, like the tracks aren't in sync with each other as much as when using RAM1.

Hopefully this gives you enough to go on. I was surprised at how good this was so far, but I have not tried any of the pitch or looping stuff at all yet. One can easily modify the sketch to try and do something along those lines. Very interested to see how this progresses.
 
Just jumping in here to say that I'm using Nic's library in my O_CT4.1 development, building a 2-deck DJ-style file player, experimenting on this branch (watch out, it's messy). I've hacked in a few things: tolerating unknown WAV metadata to find the 'data' chunk, as well as searching the 'id3' chunk for a TBPM frame for tempo. Synchronizing the playrate to my master clock works quite nicely! I've got a crude looping function working, trying to expand on that today...

I'm still hitting some hangups & crashes once in a while, maybe from SD card lag... but it sounds like @h4yn0nnym0u5e has done a lot of robustness work that's not merged in to newdigate/master?! I'd love to utilize the PSRAM for caching. I'll probably be forking the library here soon...

EDIT: here's a clip of it in action!
 
Last edited:
@graydetroit thanks for that, got it all and investigating now. There's definitely "something rotten in the State of Denmark", I'm getting intermittent crashes when playback is started, specifically during loading of the buffers with audio data. Unfortunately CrashReport is not giving a useful address, probably due to optimisation, so figuring out the cause is non-trivial.

Once that's found and fixed we can think about the predictive loading stuff.

From a timing point of view, it looks like playback is taking about 8ms to start, shared roughly equally between file open, header parsing and buffer loading. There may be some scope for optimising these, as @djphazer says the header parsing is a bit rough and ready... Again, not worth addressing while there's this bug in the playback start code.
 
Thanks @h4yn0nnym0u5e and I was also thinking if the specifics of the WAV parsing are a hangup, we could work with RAW files instead. I know WAV is more user-friendly, but I'm not really planning on having stereo sample support at first, and I was thinking of making a companion desktop app that would enable the user to drag and drop their WAV files and convert them to RAW and organize them into the uSD for them, in order to reduce that (small) overhead and because RAW files are easier for the code to work with.
 
Actually, so long as the programmer makes just a bit of effort to parse fairly basic WAV files, they're a lot easier than RAW. They contain vital information about how many channels you have, the sample rate, and whether the file is really 16-bit PCM.

It's looking like a memory leak ... sufficiently severe that I'm not quite sure how it wasn't apparent ages ago. Preferable to some real-time interrupt-related only-happens-on-a-Tuesday bug, anyway. Investigation continues...
 
OK, so, there seems to be nothing wrong with my library code after all! The problem is, the heap just isn't big enough for the playback buffers. I keep forgetting that they're specified in samples, so 2048x7 is actually 28kB per object, and we have 32 of them, so that's 896kB without taking into account all the other bits and bobs that get allocated.

I've posted my edits here (never used gists before, can't say I'm a fan on first try - wot? no branches [visible via web interface]?). The key items are:
  • use PSRAM for buffering
  • fixed how you used elapsedMillis objects to do timing: % NNN can fail badly on Teensy, it's so fast that you can re-run the "event" code several times before the millisecond timer rolls over
  • re-enabled tracks 12-16
There's some hardware debug pins so I can see timing on a 'scope, and I made it play a tune rather than banging out the one note samba. It chews quite a bit of CPU, but seems to be stable, and given the playback triggering is horribly slow (about 12ms) it actually doesn't sound too bad to my un-trained ear. Groovebox aficionados may disagree...
 
Ok, yeah not surprising about the buffer size issue, I was wondering if that might be it.

Also interesting is that the two patterns it cycles through switch the track assignments so that the sample voices are already swapping between different samples every time a pattern change occurs without a predictive/preload mechanism in place. So that's nice.

Thanks for taking the time to test this. 12ms seems less than ideal to me, so I'm not sure if I'll decide to overhaul my groovebox to use this implementation, though I very much desire to. But then again, these patterns are more of a stress test, I'm not sure if most users will often fully utilize 16 sample tracks, since my groovebox also has more sound generators than just samples (4x mono synth voices, 4x dexed with 4-voices of polyphony each, 3x fm drum voices, and eventually 1x Braids voice).

One of the features of my groovebox is a ratcheting performance feature, which allows a user to generate bpm-sync'd triggers from quarter notes all the way to really fast 1/96ppqn. I'm thinking that won't really jive too well with the buffering approach, but maybe that's due to a misunderstanding, please correct me if I'm wrong.

Maybe I'll go with a hybrid solution as I imagined before, two different sample track types instead of one. With the one-shot sample track type, I can apply some limits on file size (~100-200kb limit) and I could dedicate some space in PSRAM for lots of generally fast reads for short intervals. And then for a more flexible sample track type, I could dedicate some other space for buffered samples for doing stuff with larger samples but which maybe can't be ratcheted snare-rush style.
 
Though, with lowering the RESAMPLE_BUFFER_SAMPLE_SIZE to 512 it seems better playback wise and I haven't encountered a crash yet, less slop between various samples firing at the same time vs. when I had it at 2048. I think I could even get away with not having to use 32 playback objects maybe, which should help even more I guess.
 
Sorry, my thoughts keep pouring in instead of altogether at once. First, thank you so much again, really inspired by this progress. I was kind of dreading the current state of sampling on my box, and now I'm looking forward to seeing how far I can push this.

I hope you don't mind, I still have a couple questions:
1. Does your feature/set-start-point branch require your other fork of the Teensy Audio core library changes as well? I currently have those changes, but wasn't sure if they were necessary to support the Teensy Variable Playback changes of yours.
2. Do you still think you would be interested in supporting triggering these buffered playback objects from within an interrupt? That would be my only blocker from completely trying to overhaul my project at this point, which I am now kinda eager to do, but you mentioned how those changes would be deep so just wanted to see if you have any appetite for it.

Thanks again!
 
Whatever works for your application, really. I'm just glad someone's pushing the boundaries a bit, and willing to supply test cases. I'm under no illusions my code is bug-free... I was wondering if having several layers of directories was slowing open() down, for example. I may have a further play if and when I get time and motivation. Another thing to do would be mix heap and PSRAM buffers. All sorts of things to try.

You need to be a bit careful with setting RESAMPLE_BUFFER_SAMPLE_SIZE, especially with pitch shifting - test the limits and then set bigger / more buffers than "seem to work", or ... do the maths!

Oh, and using uclock in a simplistic manner is very likely to cause problems, because you'll be starting objects from interrupt while others are buffering from foreground code. That's a big no-no. Been there, done that, doesn't work. BUT if you (or I) manage to get the predictive slot code working, that might be triggerable from interrupt, because all the filesystem activity will have been done in foreground code.

Nothing in this library depends on any of my Audio or cores changes.

At some point I may consider making the buffered filesystem playback code triggerable from interrupt. At the moment I'm just hoping it'll make it into Teensyduino 1.60 as it is - I've put in a PR, and am awaiting results with bated breath! As such, I don't really want to muddy the waters with possible future changes, especially as what looked fairly simple to begin with ended up turning into a many-headed monster, and as such a bit liable to introduce a lot of bugs unless I re-write it nearly from scratch.
 
Ok, that makes sense. But now that I know that the Teensy Variable Playback changes don't require the Teensy Audio library changes, I am wondering if the Teensy Variable Playback changes are/can be easily interrupt safe, specifically playing AudioPlaySdResmp files from within an interrupt from uClock. If not, and if that's still too much of a headache, then I can try and just have the main loop / foreground code watch for triggers and fire them off that way perhaps. I just know that in the past when I've used a clock solution in the main loop while also having display processing and all sorts of other things, I would notice it might affect the clock timing. It's something I will think on more, but this is great so far.
 
“Easily” is almost certainly a no, though I could be wrong.

The fundamental issue is that for efficient streaming of multiple files, you need to read in quite big chunks - 4kB is a sweet spot for SD, typically. For current technology. It’s highly unlikely that you could arrange to do all necessary reads, every time, within the 2.9ms audio update. You might need many reads, and the SD card might decide to take a long time “housekeeping”. So you have to take an approach which is functional with a longer read time, which means reading plenty of data into big buffers in the foreground code, which the audio update can use gradually.

I’ve used the EventResponder library to hide this from the user. It’s possible one could hijack it such that its response runs in your uClock ISR, rather than foreground code. Then all SD access occurs at the same interrupt level and there ought to be no interference. The penalty is NO SD access from foreground code, at least while you’re playing. And maybe no other library can use EventResponder…
 
Yeah with having to sacrifice all of the PSRAM for caching samples freed up now, I can probably move all of my project files into PSRAM during a play session, rather than how I have it now which is to read/write project data to the SD card in between pattern changes in order to save on memory, since the project data is quite large. So in theory, I wouldn't need to read or write to the SD card from foreground code unless the user wants to save or load projects, in which case the playback will be stopped anyway.

In order to test this, do I just need to wrap the EventResponder response in an ISR at the same priority as uClock's? Sorry, I'm not well versed in managing / writing stuff with interrupts myself.
 
Remembered as I was putting the light out that EventResponder has an interrupt option which may be relevant to your interests. Can’t work on it today but will take a look as soon as I get a moment.
 
So I made a revision to the original sketch to try and use uClock. It works because I have a separate crude data structure tracking the trigger events, and then I'm scanning for them and firing them in the main loop. It works decently for this example, and I was able to overhaul my groovebox code in a separate branch and try this out, but with all of the other stuff in the main loop that is not as timing-critical as the sequenced music stuff, it throws off the playback quite a bit, and now I just really wish I could just trigger the objects directly from the interrupt.
 
Seems like if the buffered WAV file playback is going to be in my main loop, I have to figure out how to use the SSD1309 OLED with the u8g2 library in such a way that is non-blocking, because I think the SPI communication to the OLED is what is the main issue.

EDIT: Sorry, I didn’t fully grasp the cue/preload concept you were describing, because I was assuming you were referring to preloading only when an entire pattern changes over. But you were actually referring to swapping sample objects on a step-by-step basis. That makes sense. So as each track proceeds through its steps, it alternates back and forth between two sample objects: one which is already preloaded with a sample to be triggered inside the interrupt, and one which is preloading for the next step in the main loop.

I have a better understanding of this now and can probably try to make a sketch for this.
 
Last edited:
You must log in or register to reply here.
Back
Top