changing pitch of audio samples - TeensyVariablePlayback library

Hi moo, a quick question to see if this is what I'm looking for. I want to make a sampler that does this: a live input can be sampled for as long as a button is pressed. A pot can then change the playback speed of the sample, so that for example the sample can be downtuned and played back very slowly.

I've made simple live samplers with teensy before but changing the sample playback speed was not possible for my limited coding means. I've been checking the forums here every few months for a few years now checking to see if anyone had finally made an easy to use library for this... But just wanted to see if it will do what I'm looking for? Cheers for your work on this regardless. Ed
 
yes, Im pretty sure that will be possible... If you use a record queue to save a raw file to SD, you can then load the raw file and playback at different pitches. :p
 
Dear Moo,

is it a known problem to use TeensyVariablePlayback with LittleFS from Flash instead of the SD Card ? I have replaced the hardcoded SD.open calls with myfs.open (following the example code for LittleFS) and it does kind of work - but when using multiple samples at once it tends to hang after playing 4-5 samples.
It does NOT crash when using a single file at once.
CPU does not seem to be the issue because it works flawless when 8 Samples at once, using AudioPlayArrayResmp* instead of AudioPlaySdResmp* (so my goal is to replace the arrays from progmem with files from flash)

Any ideas ?
 
Hi Moo and users of TeensyVariablePlayback,

after getting no usable results with LittleFS on internal Flash i changed to using serialflash.h and put the chip on the audio board. This does work for around 50 -60 seconds and then crashes after playing 4 tracks with samples because of a memory leak. I am pretty sure i created this leak myself because i am not an experienced developer at all (having no real idea how pointers are used correctly and be cleared after usage etc.)

I suspect my mistake is somewhere in this file where i replaced the "File _file" etc. commands from the SD Library with "SerialFlashFile _file" with the commands of the serialflash Library.
As said, it does work for around a minute and then crashes because of no more free RAM because it is consuming more RAM with every sample it is playing.

I would be very glad if someone more experienced would have a look what i did wrong.

Thank you dear teensy community!

Code: 
#ifndef TEENSY_RESAMPLING_INDEXABLE_FILE_H
#define TEENSY_RESAMPLING_INDEXABLE_FILE_H

#include <Arduino.h>
#include <SerialFlash.h>
#include <vector>

namespace newdigate {

struct indexedbuffer {
    uint32_t index;
    int16_t buffer_size;
    int16_t *buffer;
};

constexpr bool isPowerOf2(size_t value){
    return !(value == 0) && !(value & (value - 1));
}

template<size_t BUFFER_SIZE, size_t MAX_NUM_BUFFERS> // BUFFER_SIZE needs to be a power of two
class IndexableFile {
public:
    static_assert(isPowerOf2(BUFFER_SIZE), "BUFFER_SIZE must be a power of 2");

    static constexpr size_t element_size = sizeof(int16_t);
    size_t buffer_to_index_shift;
    IndexableFile(const char *filename) : 
        _buffers(),
        buffer_to_index_shift(log2(BUFFER_SIZE)) {
            _filename = new char[strlen(filename)+1] {0};
            memcpy(_filename, filename, strlen(filename));
            _file = SerialFlash.open(_filename);
    }

    int16_t &operator[](int i) {
        int32_t indexFor_i = i >> buffer_to_index_shift;
        indexedbuffer *match = find_with_index(indexFor_i);
        if (match == nullptr) {
            if (_buffers.size() > MAX_NUM_BUFFERS - 1) {
                indexedbuffer *first = _buffers[0];
                _buffers.erase(_buffers.begin());
                delete [] first->buffer;
                delete first;
            }
            indexedbuffer *next = new indexedbuffer();
            next->index = indexFor_i;
            next->buffer = new int16_t[BUFFER_SIZE];
            size_t basePos = indexFor_i << buffer_to_index_shift;
            size_t seekPos = basePos * element_size;
            _file.seek(seekPos);
            int16_t bytesRead = _file.read(next->buffer, BUFFER_SIZE * element_size);
            #ifndef TEENSYDUINO
            if (!_file.available()){  
                _file.close();
                __disable_irq();
                _file = SerialFlash.open(_filename);
                __enable_irq();
            }
            #endif
            next->buffer_size = bytesRead;
            _buffers.push_back(next);
            match = next;
        }
        return match->buffer[i % BUFFER_SIZE];
    }

    void close() {
        if (_file)
            _file.close();

        if (_filename)
            delete [] _filename;
        _filename = nullptr;
    }
private:
    SerialFlashFile _file;
    char *_filename;
    std::vector<indexedbuffer*> _buffers;

    indexedbuffer* find_with_index(uint32_t i) {
        for (auto && x : _buffers){
            if (x->index == i) {
                return x;
            }
        }
        return nullptr;
    }
};

}

#endif
 
Hi Moo
I'm checking out the teensy variable playback library and trying to run the example using a teensy 3.2 with the audio shield. I am getting the following error.

C:\.....Arduino\libraries\TeensyVariablePlayback\src/ResamplingArrayReader.h:30: undefined reference to `external_psram_size'
collect2.exe: error: ld returned 1 exit status

I've looked at the ResamplingArrayReader.h file and there is no text saying 'external_psram_size'. I'm not totally sure what I am doing or if looking for that text would even help. Would you have any ideas on how to solve this?

Thanks
 
Hello phantomchips,

i am not sure if i can help you but afaik there is no variable or define of that name in Moo's Library. My first suspect is that your Teensy Audio Library/Teensyduino Environment is outdated?
Further, are you using any PSRAM at all ? If i remember correctly the Teensy 3 had not any option to put a PSRAM or Flash Chip.
 
Hi Positionhigh, thanks for the suggestions. I updated teensyduino while I was trying to fix it, but perhaps some of the libraries are being overwritten by the ones in my sketch folder, not sure if teensyduino installed the update into my program files/arduino library. I will take a look in the morning, it's late here. :)
As far as I know I'm not using PSRAM, tbh I'm not even sure what that is.
Thanks :)
 
Hi Nic, I just updated to v1.0.14 for a project I'm working on that uses the sampleloader example to load wav files from the sd card into the psram on teensy 4.1. Using the example, the playRaw seems to work just fine but playWav doesn't seem to be outputting any audio. Everything seems to compile and the loader looks to be printing the normal sample start/size/available messages. Any idea what might be causing this?
 
Hello otemrellik,

i am not sure if i can help you but if you have static samples, they would be best played from progmem, that seems to be the fastest method. It will also work from SPI flash. About psram, we are also looking into that at the moment.
 
hi otemrellik. i’ll have a look when I have a chance. in the meantime it’s probably best to revert to a previous working version … thanks for letting me know, much appreciated

cheerz
 
Moo said:
hi otemrellik. i’ll have a look when I have a chance. in the meantime it’s probably best to revert to a previous working version … thanks for letting me know, much appreciated

cheerz
Click to expand...
Hey Moo, one strange thing I found after posting this was that if I use playRaw instead of playWav, everything seems to work even though the files I'm loading are not raw files, they are wav files. I initially updated to the latest version because I was having crashes, unfortunately with the latest update that is still happening. This might be an issue on my end though.
 
Moo said:
github.com/newdigate/teensy-variable-playback
I started this endeavour a few years back, I needed samples played at variable playback rate.

The initial implementation was a hack but it worked okay for non-interpolated resampling, a little buggy with linear interpolation - especially in reverse. (Sounds awful without interpolation, almost like 8-bit.. linear interpolation is slightly better)

There was a lot of interest in this code and I finally had a chance to iron out the wrinkles and implement quadratic interpolation. 300 commits later and it sounds much better and its stable.

It can play from array or from uSD card file (raw or wav)

I've made it available from the Arduino library manager, just search TeensyVariablePlayback - and have a look at the examples.

I'm going to look at implementing stereo, when I get a chance next.

I've tested on teensy 4.1 and teensy 3.6

Please feel free to use, develop, improve. I would be interested to know what you think.

Cheers
Nic
Click to expand...

Hi Nic
Today I found your work! It is literally awesome.
I just started digging in examples, doing some testing. It working without problems.
Just working about stop command. I didn’t see in keyword file any and in example play sd raw too.
After loading example it starts automatically.
Literally 🤣 I need stop.
 
Love this library as well!

I think you should be able to just do:
Code: 
playSdRaw1.stop();
But I could be misunderstanding.
 
Hi @Moo,
I'm currently using a modified version of your old code, will give a try to this new one which seems to be way better. How does it perform reading from the microSD card?
 
Hi folks

I've finally managed to get a bit of time to look at asynchronous (i.e. not in the audio update interrupt) loading of sample data for this library. The existing structure doesn't really lend itself to my previous buffering approach for 1x playback, so I've done it slightly differently, but it should still be an improvement. You can find my feature development branch at https://github.com/h4yn0nnym0u5e/teensy-variable-playback/tree/feature/async-read. If you're really brave, you can even give it a go!

Implemented so far are:
  • triggering an EventResponder event per-playback object, on every audio interrupt
  • if the event finds a buffer it thinks is no longer going to be used, it tries to load it with data it thinks will be useful in the future
Broken at this point are:
  • looping, almost certainly
  • playback at speeds <0.0, i.e. in reverse
I haven't tested data sources other than SD card, and only up to 4 mono files at once, and then not rigorously.

Next steps, in no particular order, would be:
  • improving the predictive loading, so that it can cope once more with reverse playback and looping
  • further improvements so we wait for multiple buffers to become unused before a file load occurs (big loads are more efficient, up to a point)
  • optionally storing buffers in PSRAM if large amounts are needed
  • determining whether the existing compile-time setting of buffer sizes and counts actually make sense
@Moo, if you think I should start a separate thread for this, do let me know.

If anyone tests this and finds an issue that isn't on the "known broken" list above, do let me know, with the usual Forum Rule i.e. a simple complete example that anyone can compile to reproduce the problem. Please do not PM me unless you have a really good reason ... if I don't think it's a good reason, I simply won't reply. Sorry.

I have quite a lot going on at the moment, so can't promise to do any fixes or improvements on any particular timescale. I will try not to abandon this, unless it becomes clear that the target feature(s) are not attainable: my target is stable multi-file playback from currently supported filesystems, at some reasonable range of playback speeds, by using asynchronous file reads. Additional features are out of scope.

So, please, step up, give it a go and see what you think.
 
Thank you for putting in time and work on this, h4yn0nnym0u5e.
Currently i can't get your branch working for more than 0.5-1sec. but that very well might have reasons out of your scope and i will try to find it.

One question: With the code from Nics repo, we had to insert
__disable_irq();
before physical file reads and then continue with
__enable_irq();

I understand that you are also working with interrupts so it might be contradicting to try to get your code working with the same method, however that currently is the best approach i have :_)
Will let you know when i have progress.
 
M4ngu said:
Hi @Moo,
I'm currently using a modified version of your old code, will give a try to this new one which seems to be way better. How does it perform reading from the microSD card?
Click to expand...

Hi M4ngu, the more recent versions are slightly worse for performance, I added abstractions for SerialFlash and sacrificed some performance. I have been dreaming of using 'functors' for the best of both worlds but its not top on my list at the moment. There is also an improvement to the interpolations that can be massively improved I recon.

Sorry for such a late response, I didn't get any notification....
 
positionhigh said:
Thank you for putting in time and work on this, h4yn0nnym0u5e.
Currently i can't get your branch working for more than 0.5-1sec. but that very well might have reasons out of your scope and i will try to find it.

One question: With the code from Nics repo, we had to insert
__disable_irq();
before physical file reads and then continue with
__enable_irq();

I understand that you are also working with interrupts so it might be contradicting to try to get your code working with the same method, however that currently is the best approach i have :_)
Will let you know when i have progress.
Click to expand...

I've just pushed up some changes which catch some NULL pointer cases: it seems to be a lot more stable now, so I'd recommend giving that a go.

I've tested with different speeds including reverse play and it's "working", but there will still be cases at the moment where file reads during audio update interrupts will still occur. I think this is only looping and reverse playback - I'm planning to lock down the loop start and/or end buffers, and take playback direction into account on the "predictive load" events. You could try dispensing with the interrupt disable/enable while doing "physical file reads" (guessing this means reads from SD of non-audio data, e.g. text, images etc.), but I'd only expect it to be robust for non-looped forward playback right now.

The technique I'm using is to use EventResponder. Audio updates trigger an event (every update), and when it's acted on it looks to see if a buffer has been freed, and if so, loads it with the "next" audio data. This action is within yield(), so very much not under interrupt and should not interfere with application access to other open files. One of my tests is to play back from a single file at 4 different speeds at once, using 4 separate AudioPlaySdResmp objects, seems to be working fine. Please note though that my current definition of "working fine" is "doesn't actually crash", and even then my tests are very limited in scope, which I why I only just encountered the NULL pointer issues! Things like audio quality and latency are not going to get looked at just yet.
 
I understand you do not have much free time but just to let you know - you can test/experience our project with just a bare teensy 4.1 and your PC connected via the onboard MicroUSB and a Webbrowser.
You even will hear audio over USB in the Webbrowser, see the "screen" and can control everything by PC keyboard.
Let me know if you like to get any help or assistance.
 
h4yn0nnym0u5e said:
Hi folks
@Moo, if you think I should start a separate thread for this, do let me know.
Click to expand...

Hi h4yn0nnym0u5e,

Sorry for a long period of silence from me.

Reading the SD card during an interrupt is less than ideal so your async method could avoid that, and thats a great thing.

I would love to, but I don't have much time to write 'recreational' code, at the moment. I'm coding up to 40 hours a week at work usually. I do find some time to code, sometimes an hour after work, but I have to prioritise what I do with that time. so I try to find the intersection of the most useful/necessary thing and the thing I enjoy doing most. (and I tend to go down rabbit holes quite often...). And occasionally I find time to binge 'recreational' code.

At the moment, 'recreationally', I'm mostly working on my polyphony/sampler/sequencer. Im having a lot of fun writing it and many cool elements are all coming together. It is a modular music machine in teensy from the ground up , decoupling sequencers(step-sequencers, midi files/loops) from voices(sine,wav,raw,wavetable,...)....

The teensy-variable-playback library needs so many improvements at the moment - I don't think its even working at the moment with SD! as PositionHigh said - he needed to manually insert disable and enable irq methods. Theres so much stuff that needs attention/improvement/efficiency/performance, and I am planning on spending some time soon on addressing that

So if you can make it stable with no pitch changing, ...at some point in the future, when I've improved my variable-playback, I will try to find time to apply pitch changing to your code if I can...

let us know how far you've progressed on the Async code. Its very interesting, I've tried to think how I could apply this concept to my library and I havent figured it out... yet.... :_)
 
Diodac said:
Hi Nic
Today I found your work! It is literally awesome.
I just started digging in examples, doing some testing. It working without problems.
Just working about stop command. I didn’t see in keyword file any and in example play sd raw too.
After loading example it starts automatically.
Literally &#55358;&#56611; I need stop.
Click to expand...

Aw its lovely to hear that! thanks man, I don't know how many silly hours of my life I lost to this code!!!! :) LOL

PS> Pull requests to add keywords like close, absolutely welcome my bro.... (or maybe I'll add it if i remember when I there next)
 
You must log in or register to reply here.
Back
Top