PDA

View Full Version : Granular Synthesis with Teensy and Audio Adaptor



receter
07-30-2015, 07:45 PM
I want to build a granular synth that I can control via MIDI. First I wanted to build it with an Arduino DUE, but then I found the Teensy. It got my attention because of the 16bit ADC.

I already bought a Teensy3.1 and the Audio Adaptor Board. Are there already some granular synth projects for Teensy you know of? I tried google but did not find any. I was just wondering because it seems to me that this is the perfect board for it.

Thanks!

PaulStoffregen
07-30-2015, 08:25 PM
I've not heard of anyone doing this yet. I hope you'll share your progress here.

The Teensy Audio Library processes audio in 128 sample blocks, which is approx 3 ms at 44.1 kHz sample rate. If your grain size is a multiple of 3 ms, this should be pretty achievable. If you're looking to get down to under 3 ms timing, you'll have a lot of really tough programming to do!

receter
07-31-2015, 07:11 AM
Yea, I will totally share my progress and code here. 3ms should be fairly enough I think. The Audio System Design Tool looks pretty handy, I like how every item has such a detailed explanation with photos and circuitry. I guess the audio input would be as easy as connecting a adc to a queue and write some loop that writes the queue content to memory, isn't it?

I am already looking forward to get my boards next week.

MacroMachines
08-01-2015, 09:03 AM
Here is something I quickly tried out which copies the incoming audio to a buffer and plays it back with the arbitrary waveform generator. In effect it currently lets you crudely pitch shift the audio with the built in trimpot that can be installed on the audio board. I started to experiment with using a sine wave to envelope two different arbitrary waveform generators, but then I filed it away for later use. The two sine wave windows are 180 degrees out of phase and triggered each time the buffer fills(obviously this could and should be played with). If you come up with any advancement from this please share as I am curious what could be possible.



// Nicholas C. Raftis III arbitrary waveform pitch shifting/granular test v0.01
// use the optional trimpot on the audio board to adjust playback pitch of wavetables
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>

AudioInputI2S i2s1; //xy=57,61
AudioRecordQueue queue1; //xy=176,60
AudioSynthWaveformSine sine1; //xy=305,224.75
AudioSynthWaveformDc dc1; //xy=305,258.75
AudioSynthWaveform waveform1; //xy=451,186.75
AudioMixer4 mixer1; //xy=456,239.75
AudioEffectMultiply multiply1; //xy=599,202.75
AudioSynthWaveformSine sine2; //xy=691.0000095367432,340.0000066757202
AudioSynthWaveformDc dc2; //xy=692.2500095367432,376.2500066757202
AudioSynthWaveform waveform2; //xy=843.5000057220459,298.7500009536743
AudioMixer4 mixer2; //xy=853.5000133514404,368.7500066757202
AudioEffectMultiply multiply2; //xy=998.5000133514404,337.5000066757202
AudioMixer4 mixer3; //xy=1132.2500114440918,192.5
AudioOutputI2S i2s2; //xy=1257.75,192.74999618530273
AudioConnection patchCord1(i2s1, 0, queue1, 0);
AudioConnection patchCord2(sine1, 0, mixer1, 0);
AudioConnection patchCord3(dc1, 0, mixer1, 1);
AudioConnection patchCord4(waveform1, 0, multiply1, 0);
AudioConnection patchCord5(mixer1, 0, multiply1, 1);
AudioConnection patchCord6(multiply1, 0, mixer3, 0);
AudioConnection patchCord7(sine2, 0, mixer2, 0);
AudioConnection patchCord8(dc2, 0, mixer2, 1);
AudioConnection patchCord9(waveform2, 0, multiply2, 0);
AudioConnection patchCord10(mixer2, 0, multiply2, 1);
AudioConnection patchCord11(multiply2, 0, mixer3, 1);
AudioConnection patchCord12(mixer3, 0, i2s2, 0);
AudioControlSGTL5000 sgtl5000_1; //xy=1415.75,474.75


int16_t buffer1[256];
int16_t buffer2[256];

int tableFreq = 172;


void setup(){

AudioMemory(64);

sgtl5000_1.enable();
sgtl5000_1.inputSelect(AUDIO_INPUT_LINEIN);
sgtl5000_1.volume(0.8);
sgtl5000_1.lineInLevel(0);
sgtl5000_1.lineOutLevel(13);

waveform1.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
waveform1.arbitraryWaveform(buffer1, 344);
waveform2.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
waveform2.arbitraryWaveform(buffer2, 344);

sine1.amplitude(0.5);
sine2.amplitude(0.5);
dc1.amplitude(0.5);
dc2.amplitude(0.5);

queue1.begin();

}

void loop(){
tableFreq = map(analogRead(15), 0, 1024, 172/2, 172*2);

if(queue1.available() >= 2){ // input buffer
memcpy(buffer1, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer2, queue1.readBuffer(), 256);
queue1.freeBuffer();

AudioNoInterrupts();
waveform1.phase(0);
waveform1.frequency(tableFreq);
sine1.phase(0);
sine1.frequency(tableFreq);

waveform2.phase(180);
waveform2.frequency(tableFreq);
sine2.phase(180);
sine2.frequency(tableFreq);
AudioInterrupts(); }
}

MacroMachines
08-01-2015, 09:16 AM
Another slight variation that has a kindof interesting digital distortion effect




// Nicholas C. Raftis III granular teensy test v0.04

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>

// GUItool: begin automatically generated code
AudioInputI2S i2s1; //xy=55,38
AudioRecordQueue queue2; //xy=186,38
AudioSynthWaveformSine sine1; //xy=302.85713958740234,79.99999904632568
AudioSynthWaveform waveform1; //xy=312.2856979370117,40.42856788635254
AudioEffectMultiply multiply1; //xy=455.7142906188965,45.71428680419922
AudioSynthWaveformSine sine2; //xy=308.57141494750977,162.85712099075317
AudioSynthWaveform waveform2; //xy=308.5714416503906,128.5714235305786
AudioEffectMultiply multiply2; //xy=452.8571363176618,132.85714135851177
AudioSynthWaveformSine sine3; //xy=310.00000762939453,270.0000009536743
AudioSynthWaveform waveform3; //xy=304.2857131958008,231.4285593032837
AudioEffectMultiply multiply3; //xy=451.4286422729492,245.71429538726807
AudioSynthWaveformSine sine4; //xy=297.1428527832031,361.42855644226074
AudioSynthWaveform waveform4; //xy=302.8571472167969,321.428564786911
AudioEffectMultiply multiply4; //xy=467.14283752441406,329.9999837875366
AudioMixer4 mixer1; //xy=628.5714378356934,187.14285564422607
AudioPlayQueue queue1; //xy=720.4285888671875,361.2857074737549
AudioOutputI2S i2s2; //xy=757.4286193847656,189.71427631378174
AudioConnection patchCord1(i2s1, 0, queue2, 0);
AudioConnection patchCord2(sine4, 0, multiply4, 1);
AudioConnection patchCord3(sine1, 0, multiply1, 1);
AudioConnection patchCord4(waveform4, 0, multiply4, 0);
AudioConnection patchCord5(waveform3, 0, multiply3, 0);
AudioConnection patchCord6(waveform2, 0, multiply2, 0);
AudioConnection patchCord7(sine2, 0, multiply2, 1);
AudioConnection patchCord8(sine3, 0, multiply3, 1);
AudioConnection patchCord9(waveform1, 0, multiply1, 0);
AudioConnection patchCord10(multiply3, 0, mixer1, 2);
AudioConnection patchCord11(multiply2, 0, mixer1, 1);
AudioConnection patchCord12(multiply1, 0, mixer1, 0);
AudioConnection patchCord13(multiply4, 0, mixer1, 3);
AudioConnection patchCord14(mixer1, 0, i2s2, 0);
AudioControlSGTL5000 sgtl5000_1; //xy=744.5713729858398,254.42857027053833
// GUItool: end automatically generated code

int16_t buffer[1024];
int tableFreq = 172;


void setup(){
AudioMemory(64);
sgtl5000_1.enable();
sgtl5000_1.inputSelect(AUDIO_INPUT_LINEIN);
sgtl5000_1.volume(0.8);
sgtl5000_1.lineInLevel(0);
sgtl5000_1.lineOutLevel(13);
queue2.begin();
waveform1.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
waveform1.arbitraryWaveform(buffer, 344);
waveform2.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
waveform2.arbitraryWaveform(buffer+256, 344);
waveform3.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
waveform3.arbitraryWaveform(buffer+512, 344);
waveform4.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
waveform4.arbitraryWaveform(buffer+768, 344);
sine1.amplitude(0.8);
sine2.amplitude(0.8);
sine3.amplitude(0.8);
sine4.amplitude(0.8);
}

void loop(){
tableFreq = map(analogRead(15), 0, 1024, 172, 344);

if(queue2.available() >= 4){ // input buffer
memcpy(buffer, queue2.readBuffer(), 256);
AudioNoInterrupts();
queue2.freeBuffer();
waveform1.phase(0);
waveform1.frequency(tableFreq);
sine1.phase(0);
sine1.frequency(tableFreq);
AudioInterrupts();

memcpy(buffer+256, queue2.readBuffer(), 256);
AudioNoInterrupts();
queue2.freeBuffer();
waveform2.phase(0);
waveform2.frequency(tableFreq);
sine2.phase(0);
sine2.frequency(tableFreq);
AudioInterrupts();

memcpy(buffer+512, queue2.readBuffer(), 256);
AudioNoInterrupts();
queue2.freeBuffer();
waveform3.phase(0);
waveform3.frequency(tableFreq);
sine3.phase(0);
sine3.frequency(tableFreq);
AudioInterrupts();

memcpy(buffer+768, queue2.readBuffer(), 256);
AudioNoInterrupts();
queue2.freeBuffer();
waveform4.phase(0);
waveform4.frequency(tableFreq);
sine4.phase(0);
sine4.frequency(tableFreq);
AudioInterrupts();

}
}

receter
08-10-2015, 09:18 AM
Thanks! I'm sure this will help me getting started. I love it to have some code as a starting point.

My boards just arrived this morning, I will play around with your code this week and keep you posted what I have come up with.

MacroMachines
08-12-2015, 11:59 AM
I should mention that they may not sound very much like granular in their current form, the main thing that would need to happen to get a more granular type effect would be to change the timings of the windows/retriggering for each grain, maybe add some randomness to that timing, and possibly find a better and more consolidated way to play the individual grains.. possibly a grain function or even a new audio object entirely

MacroMachines
08-12-2015, 12:02 PM
Also worth looking into is the MOZZI library, I really would love to adapt some of their objects into the teensy audio library.

receter
08-14-2015, 10:13 PM
Thanks, I will definitely take a look into MOZZI later, looks quite comprehensive. I already have your code up and running, sounds great :-) The next thing I want to try is to make an envelope for the grains. Then I want to build a button where I can record some seconds of audio. The grains will then be picked from this recorded audio sample in different ways. For the beginning I will add an encoder that can select the origin of the grains content in the sample. A second encoder could control the frequency in which the grains are repeated. And a third could control the length of the grain. That would be already a nice thing to have!

nagual
08-15-2015, 09:08 AM
Are you trying to make it sample an incoming stream?
Also it would be great to detect zero crossings so there is no useless aliasing, it might sound better than to use fade in and out on the window edges. I havent looked at the code yet, but im no code god.

PaulStoffregen
08-16-2015, 11:00 PM
I'm curious to hear how Mozzi compares.

nagual
08-18-2015, 10:05 AM
Resampling should prove useful if it would be musical, like if you choose a note, the selected grain would play back at that rate.

drjohn
08-18-2015, 03:03 PM
For those wanting more examples of granular synthesis in a finished (and IMHO rad) product:

http://www.bastl-instruments.com/instruments/microgranny/

Oh look, there is well commented code! Not the latest version with all the 2.4 features, but still...readable, commented code:

https://github.com/bastl-instruments/microGranny2

Notable comments:

* based on WaveRP library Adafruit Wave Shield - Copyright (C) 2009 by William Greiman
* -library heavily hacked - BIG THX https://code.google.com/p/waverp/

* -thanks for understanding basics thru Mozzi library http://sensorium.github.io/Mozzi/
* -written in Arduino + using SDFat library

receter
08-19-2015, 11:04 AM
@MacroMachines
While reading your code I noticed this part:



memcpy(buffer1, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer2, queue1.readBuffer(), 256);
queue1.freeBuffer();


This copies the output of queue1.readBuffer() to the buffer which holds the arbitrary waveform. As I understand it the arbitrary waveform needs 256 samples = 512bytes, so this fills only half of the arbitrary waveform. Shouldn't it be more like this, or do I miss something?



if(queue1.available() >= 4){ // input buffer

memcpy(buffer1, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer1+256, queue1.readBuffer(), 256);
queue1.freeBuffer();

memcpy(buffer2, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer2+256, queue1.readBuffer(), 256);
queue1.freeBuffer();

}


@drjohn
Thanks, I already stumbled upon this project but I did not take a closer look at it yet. Do you know if it it supports overlapping grains?

@nagual
I want to be able to record some seconds of audio input to memory and take this as the origin for the grain content. Zero crossing is sure a good thing to avoid clipping without the need of an envelope.

MacroMachines
08-19-2015, 11:22 AM
I think I posted 2 versions. I believe I did do what you mention in one of my versions, this was an hour worth of toying around total so I haven't gotten very far as of yet. I have big plans for this project after my current product is finished. I have already added some new useful functionality into the playSDraw object, and plan to extend the wavetable and sound file objects as well while I make progress on my new product, and will share my results openly. I have studied and created a few detailed granular synthesis engines in puredata and maxmsp/gen~ and I know pretty well how it all works in theory if you want any help or pseudo code.


@MacroMachines
While reading your code I noticed this part:



memcpy(buffer1, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer2, queue1.readBuffer(), 256);
queue1.freeBuffer();


This copies the output of queue1.readBuffer() to the buffer which holds the arbitrary waveform. As I understand it the arbitrary waveform needs 256 samples = 512bytes, so this fills only half of the arbitrary waveform. Shouldn't it be more like this, or do I miss something?



if(queue1.available() >= 4){ // input buffer

memcpy(buffer1, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer1+256, queue1.readBuffer(), 256);
queue1.freeBuffer();

memcpy(buffer2, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer2+256, queue1.readBuffer(), 256);
queue1.freeBuffer();

}



@Paul I love the teensy so much, thank you for this beautiful tool! I am checking out Mozzi and so far it seems to have a few more creative synthesis tools, whereas teensy audio lib has more utilitarian functions. I also notice that much of the input variables in teensy audio lib functions are float based, where Mozzi is all fixed point based. I was shocked how good the PWM output sounds right out the gate. Ideally I plan to very soon study both libraries in depth and likely port some of the useful stuff over to teensy audio lib. Off topic: I also have made my current product splitting the programmer from the Freescale and core and it works great!

receter
08-19-2015, 11:33 AM
What is the best way too loop audio data with the Teensy Audio Library?

I found this 3 Methods:

AudioPlayMemory
I guess I can pass an array of 16bit samples to the play() function and it starts playing. What want to know is how to immediately play it again when it reaches the end.
I could check for isPlaying() in the loop() but this wouldn't be 100% perfectly on time right? Is there some interrupt or something that is called after lets say 128 samples were played?
Further I assume that the data has to be a multiple of 128 samples as the library processes audio in 128 sample blocks?

AudioPlayQueue
Looks like the most efficient way, but how can I add more than 128 samples to the queue? How do I know when it has been played?

AudioSynthWaveform / WAVEFORM_ARBITRARY
This is how @MacroMachines did it but it gets quite complex if longer waveforms/loops are needed. For every 6ms I need to create another AudioSynthWaveform with an envelope and mix all together.

I took all my information from the Audio Design Tool, is there another place where the Audio Library is documented in more detail?

receter
08-19-2015, 11:50 AM
That sounds great, looking forward to it. I also read a lot about granular synthesis and I'm so excited to put together something that works.

It's the same in both versions, I think you may have accidentally mixed samples with bytes. Your buffer array from the second snippet (int16_t buffer[1024]; ) has 2048bytes and there are four 256 pointer increments were you write 256 bytes which means 1024 bytes are filled.

receter
08-19-2015, 07:17 PM
@MacroMachines

My code was wrong, I think it needs to be like this:

Pointer increments are 16bit as it is a 16bit type. The value for memcpy is in bytes which results in 256.



memcpy(buffer1, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer1+128, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer2, queue1.readBuffer(), 256);
queue1.freeBuffer();
memcpy(buffer2+128, queue1.readBuffer(), 256);
queue1.freeBuffer();

mxxx
08-21-2015, 08:32 AM
What is the best way too loop audio data with the Teensy Audio Library?

I found this 3 Methods:

[...]

i suppose/hope the recent SPIRAM / "memoryboard" add-ons should be useful for this kind of stuff.

i started playing around a bit with effect_delay_ext, trying to turn it into a granulator object; will have to see how that goes. it'll require lots of access per update, obviously, even more when interpolating the samples.

receter
08-23-2015, 08:33 PM
My recent experiments are with PlaySdRaw. I bought a microSDHC card and modified the PlaySdRaw class to loop the data. I now have a PlayLoopSdRaw class where I can loop a sample from the SD card and set the length and position that should be looped in the raw file.

It does not have an envelope, anti aliasing or zero-detection yet but it sounds good already. I will post the class later or tomorrow.

I will run some tests on how many loops can be played and mixed simultaneously. How much faster is the SPIRAM / "memoryboard"?

mxxx
08-24-2015, 06:31 AM
How much faster is the SPIRAM / "memoryboard"?

i don't have any numbers re speed (probably Paul or Frank will be able to tell you more). but as a buffer thing those ICs have a number of advantages vs microsd. random access, obviously; no 512 byte sectors, no delay when writing.

PaulStoffregen
08-24-2015, 12:41 PM
How much faster is the SPIRAM / "memoryboard"?

It's dramatically faster than SD cards.... and that's without much SPI FIFO optimization in the code yet!

If you're playing more than 2 files simultaneously, or even 2 from a low quality SD card, you'll have much better luck with the SPI flash & ram chips.

Testing so far is looking like about 11 to 15 simultaneous mono streams can be supported. If you use the delay line object, feeding data into the RAM counts as 1 stream, and each delay tap counts as another stream. If you're just playing raw data from SPI flash, that counts as 1 stream. Of course, if you create several raw player objects reading from the flash chip, each counts as 1 stream.

SD cards are terribly slow. Nearly all cards have very substantial access latency, at least in SPI mode. From the moment the SD library asks the card for data until the moment the card returns the "I'm ready" token is often almost as long as reading the actual 512 bytes of data, which makes SD card data transfer effectively half the speed it could have been, if the card didn't spend so much time processing commands.Add on top of that FAT filesystem overhead. At the end of each allocation cluster, the SD library needs to read another sector from the FAT. That doesn't happen very often, but when it does, two 512 byte sectors need to be read.

For glitchless audio, the worst case access time is what matters. The SPI flash have fast perfectly consistent read access times. So do the RAM chips for read and write. The time taken is *always* the same, a fixed (and short) number of SPI clock cycles. Compare that with SD cards, where you have to send a request and then the card is busy for some unknown (and lengthy) amount of time.

My hope is the SD card latency will improve when we have a future Teensy with 4-bit SDIO mode. Such a future Teensy will also have more RAM, so a future version of the SD library might use more RAM for caching. I've been told multi-sector reads also suffer only short latency between sectors, so when we have more RAM for caching, maybe a future version of the library might allocate larger than 1 sector caching blocks (which will eat up quite a bit of RAM).

But for SPI mode access, those little flash and ram chips are vastly superior for audio.

receter
09-02-2015, 09:41 PM
That sounds great it's exactly what I will need! What is the "memoryboard"? Ist it some sort of shield for the teensy? The SPIRAM is a chip like this I guess: link (http://www.digikey.com/product-highlights/us/en/microchip-sram-nvsram/3142)? Do you have any recommendations?

For my prototype 2 files simultaneously are just fine to start with. I made a little example where I can loop from the SD card and set the length and position of the loop with two encoders. It also contains the PlayLoopSdRaw I wrote about earlier. The loop length and position are both set in blocks, this was the easiest way for me to implement. The same is true for why it only reads 256 bytes from the SD card per update.

I did not check if it still works when two files are played simultaneously.

PaulStoffregen
09-02-2015, 10:00 PM
What is the "memoryboard"? Ist it some sort of shield for the teensy?

Yes, exactly.

https://www.youtube.com/watch?v=d80d1HWy5_s

Frank's Memoryboard appears at 2:45 in the video. Since this video was recorded, the Memoryboard was tested and works great. It's supported by the latest version of the audio library, with this object:

http://www.pjrc.com/teensy/gui/?info=AudioEffectDelayExternal

PaulStoffregen
09-02-2015, 10:01 PM
Of course, if you're using only short grains, perhaps a short delay using the Teensy's memory could be enough? You can get up to about 0.5 second with the built-in memory.

MacroMachines
01-13-2016, 08:59 AM
i suppose/hope the recent SPIRAM / "memoryboard" add-ons should be useful for this kind of stuff.

i started playing around a bit with effect_delay_ext, trying to turn it into a granulator object; will have to see how that goes. it'll require lots of access per update, obviously, even more when interpolating the samples.

HAHA like minded indeed.. so yeah I just spent an hour looking at the delay object code with the intent of doing just this. I would love to collaborate with you on that granular delay line object if you are willing to post or send me what you have done thus far? I have made so many max and puredata granular patches from scratch and I can confidently say I have a full understanding of all granular techniques (and I have a couple favorites that are low cpu and sound amazing). I just am learning the ropes of navigating library design and C++ Object creation.

receter
01-13-2016, 11:08 AM
HAHA like minded indeed.. so yeah I just spent an hour looking at the delay object code with the intent of doing just this. I would love to collaborate with you on that granular delay line object if you are willing to post or send me what you have done thus far? I have made so many max and puredata granular patches from scratch and I can confidently say I have a full understanding of all granular techniques (and I have a couple favorites that are low cpu and sound amazing). I just am learning the ropes of navigating library design and C++ Object creation.

I would love to cooperate with you, I can upload all I have to Github. Do you have any of your patches available online?

PaulStoffregen
01-13-2016, 03:50 PM
I'm curious to see how this develops. I don't have much (well, really any) experience with granular synthesis. But the request keeps coming up to support this in the audio library. I might be able to help a bit on the Teensy coding side, but for this to come together well anytime soon, it really depends on you guys for the knowledge and experience and testing of the granular synthesis side....

MacroMachines
01-13-2016, 06:11 PM
I'm curious to see how this develops. I don't have much (well, really any) experience with granular synthesis. But the request keeps coming up to support this in the audio library. I might be able to help a bit on the Teensy coding side, but for this to come together well anytime soon, it really depends on you guys for the knowledge and experience and testing of the granular synthesis side....

I am trying to think of the best way to structure a flexible granular object for this:
at the core, a grain is a small section of an audio file through a window
(I believe you have a hanning array in the FFT already? those work great)
from there you have the option to add randomness into the buffer read position offset of each grain.
parameters for grain length, number of grains, either trigger the grains to play at random times for a very "grainy" grainular
or use a phase accumulator to drive through the windows and evenly spaced grains
which would be an easy modification/merger of the wavetable oscillator object and file playback objects. I had started to attempt this with what is already available using the queue into a larger buffer that a few wavetables play back through, but 256 is not quite enough samples. This approach is more smooth and allows for extremely clean time scrubbing when there are 4-6 grains evenly spaced along the phase ramp.
if you have puredata, or are willing to download it, I can upload one of my patches with comments showing my favorite grain styles.
for now here is a link to a nice pure data tutorial on how to build a basic phase ramp driven grain method to get some ideas:
http://pd-tutorial.com/english/ch03s07.html

if you have any questions or want me to post some additional media I will gladly do so, this would be a RAD teensy object that I am currently trying my best to grow the skill to create, but I am sure someone with more experience would be able to do it cleaner/faster.

MacroMachines
01-17-2016, 09:26 AM
Here is the max "chopper" code I was trying to port:


/*
Waveset chopper / repeater

This program divides the input into segments, and plays these segments back.
It could be seen as a time-domain, granular form of analysis/resynthesis.

The program contains a recording section,
which stores grains into a Data object (segment_data)
and a playback section,
which selects and plays these grains one-by-one

The grains are not enveloped;
instead the input is segmented at points where the signal is rising and crosses zero

A positive zero-crossing means that:
a: previous sample is less than zero
b: next sample is greater than zero

For pure sounds a segment corresponds to one or more wavecycles,
but for complex sounds it can be somewhat stochastic.

RECORDING:

Since waveforms rarely cross zero at an exact sample location,
the actual crossing is somewhere between a and b.
The program estimates this sub-sample crossing phase (and stores it in offset_data)
It also stores the sub-sample accurate segment length (in length_data)

The segment_data recorded includes the sample just before the first crossing,
and the sample just after the last, in order to contain both actual crossings.
I.e. each captured segment looks like [a1, b1, ... b2 a2 ... a3, b3]

When a segment finishes recording, a new segment is chosen to write into (write_segment)

PLAYBACK:

The playback section is continuously playing a segment (play_segment)

Playback includes additional calculations,
to ensure the sub-sample phase offset is used and retained between segments

When the segment playback is done (possibly after several repeats),
a new segment is selected according to the current strategy (play_mode)


Graham Wakefield 2012
*/


// the segment storage (each segment on its own channel):
Data segment_data(10004, 64);
// the length of each segment (in samples):
Data length_data(64, 1);
// each segment is also offset slightly (sub-sample phase delay):
Data offset_data(64, 1);
// each segment also stores it's average energy (root-mean square):
Data rms_data(64, 1);

// set to zero to disable new segment capture:
Param capture(1, min=0, max=1);
// how many zero crossings per segment:
Param crossings(1, min=1);
// the minimum & maximum length of a segment:
Param max_length(10000, min=16, max=10000);
Param min_length(100, min=16, max=10000);
// how many times a segment is played back:
Param repeats(1, min=1);
// hold the current playback segment:
Param hold(0, min=0, max=1);
// choose the strategy to play back grains:
Param playmode(0, min=0, max=4);
// choose how to select playback rates/pitches:
Param pitchedmode(0, min=0, max=4);
// playback frequency for pitchedmode enabled:
Param freq(220, min=0);
// playback rate for pitchedmode not enabled:
Param rate(1, min=0);

// the segment currently being written to:
History write_segment(1);
// the number of samples since the last capture:
History write_index(0);
// the number of rising zero-crossings since the last capture:
History crossing_count(0);

// the segment currently being played:
History play_segment(0);
// the sample index of playback:
History play_index(0);
// the length of the playing segment:
History play_len(0);
// the offset of the playing segment:
History play_offset(0);
// the loudness of the playing segment:
History play_rms(0.1);
// used to create smooth overlaps
History prev_input;
// used to accumulate the segment energy total:
History energy_sum;

// the total length of all segments
History total_length;

// the number of segments:
num_segments = channels(segment_data);



// RECORDING SECTION:

// DC blocking filter used to remove bias in the input:
unbiased_input = dcblock(in1);
// accumulate energy:
energy_sum = energy_sum + unbiased_input*unbiased_input;

// update write index:
write_index = write_index + 1;
// always write input into current segment:
poke(segment_data, unbiased_input, write_index, write_segment);

// detect rising zero-crossing:
is_crossing = change(unbiased_input > 0) > 0;
// capture behavior is triggered on the rising zero-crossing:
if (is_crossing) {

// if the segment is too long,
if (write_index > max_length) {
// reset the counters
crossing_count = 0;
write_index = 0;

} else {
// count rising zero-crossings in this segment:
crossing_count = crossing_count + 1;

// decide whether the segment is complete:
// only when capture is enabled
// only when enough zero-crossings have occurred
// only when enough samples have elapsed
// only when not too many samples have elapsed
is_complete = (capture
&& crossing_count >= crossings
&& write_index >= min_length);
if (is_complete) {

// at what theoretical sample index did it cross?
// estimate as linear intersection:
offset = prev_input / (prev_input - unbiased_input);

// compare the previous offset:
prev_offset = peek(offset_data, write_segment, 0);

// store segment length:
// adjusted for the fractional component
// minus one for the extra wrapping sample (a,b,...b,a,...,a,b)
len = write_index + offset - prev_offset - 1;
// update total length:
prev_length = peek(length_data, write_segment, 0);
total_length = total_length - prev_length + len;
// store new length:
poke(length_data, len, write_segment, 0);

// store segment energy:
// (root mean square, over number of samples measured)
rms = sqrt(energy_sum / floor(len));
poke(rms_data, rms, write_segment, 0);

// reset counters:
crossing_count = 0;
energy_sum = 0;

// switch to a new segment:
write_segment = wrap(write_segment + 1, 0, num_segments);
// don't write into what is currently playing:
if (write_segment == play_segment) {
write_segment = wrap(write_segment + 1, 0, num_segments);
}

// store the new offset:
poke(offset_data, offset, write_segment, 0);

// write the previous & current (a,b) into the new segment:
poke(segment_data, prev_input, 0, write_segment);
poke(segment_data, unbiased_input, 1, write_segment);
write_index = 1;
}
}
}

// remember previous input:
prev_input = unbiased_input;



// PLAYBACK SECTION:

r = rate;
// update playback index:
if (pitchedmode < 1) {
// no change

} else if (pitchedmode < 2) {
// ascending:
d = play_index / play_len;
r = rate * max(1, d);

} else if (pitchedmode < 3) {
// descending:
d = ceil(play_index / play_len);
r = rate / max(1, d*d);

} else {
// try to play back at a chosen frequency
// (compensating for estimated original sample frequency)
r = freq * play_len / (samplerate * crossings);
}
// update playback index:
play_index = play_index + r;
// actual play index needs to stay within len:
// (can be fun to use wrap, fold or clip here)
actual_play_index = wrap(play_index, 0, play_len);

// play the current segment waveform:
// (offset by the waveform zero-crossing position)
out1 = peek(segment_data, play_offset + actual_play_index, play_segment, interp="linear");

// switch to a new segment?
if (play_index >= play_len * floor(repeats)) {

// reset to the current actual play position
play_index = actual_play_index;

if (!hold) {
// move to a new segment
// some alternatives...
if (playmode < 1) {

// play in forward sequence
play_segment = wrap(play_segment + 1, 0, num_segments);
// caveat: don't play what is currently being written:
if (write_segment == play_segment) {
play_segment = wrap(write_segment + 1, 0, num_segments);
}

} else if (playmode < 2) {

// play in reverse sequence
play_segment = wrap(play_segment - 1, 0, num_segments);
// caveat: don't play what is currently being written:
if (write_segment == play_segment) {
play_segment = wrap(write_segment - 1, 0, num_segments);
}

} else if (playmode < 3) {

// choose direction by random walk:
direction = sign(noise());
play_segment = wrap(play_segment + direction, 0, num_segments);
// caveat: don't play what is currently being written:
if (write_segment == play_segment) {
play_segment = wrap(write_segment + direction, 0, num_segments);
}

} else if (playmode < 4) {

// choose randomly:
direction = 1 + ceil(num_segments * (noise() + 1)/2);
play_segment = wrap(play_segment + direction, 0, num_segments);
// caveat: don't play what is currently being written:
if (write_segment == play_segment) {
play_segment = wrap(write_segment - 1, 0, num_segments);
}

} else {

// play most recently recorded:
play_segment = wrap(write_segment - 1, 0, num_segments);

}

// get the new playback length
play_len = peek(length_data, play_segment, 0);
// get the new playback offset
play_offset = peek(offset_data, play_segment, 0);
// and the new playback loudness
play_rms = peek(rms_data, play_segment, 0);
}
}

// show what's actually happening:
out2 = write_segment;
out3 = play_segment;
out4 = play_len;
out5 = play_index / play_len;
out6 = play_rms;
out7 = total_length;

PaulStoffregen
01-17-2016, 09:48 AM
There's an important distinction between analysis of live input to grains versus simply playing already-prepared grains.

Being able to play them first would be the sane path....

MacroMachines
01-17-2016, 09:58 AM
what i propose is a simpler version of the above with:

recording into a ram buffer, like delay line but can be frozen to keep from updating
creates a map of zero crossing points of valid "wavelets", must cross zero twice so there is a full wave cycle
playback of a selected wavelet looping like a wavetable oscillator with pitch ratio using linear interpolation
selection of current wavelet waits until it is finished with the active playing wavelet to update, so there is never a click because they all happen at zero cross

MacroMachines
01-17-2016, 10:01 AM
I was under the impression the limitations of playback from spi flash or especially sd card make it unrealistic to play back grains from them reliably at the speed needed for smooth audio.

PaulStoffregen
01-17-2016, 10:20 AM
SD cards are too slow.

SPI flash or SPI ram is limited to bandwidth of a couple megabytes/sec. That's plenty fast enough for several overlapping samples/grains, but not dozens of them.

MacroMachines
01-17-2016, 11:13 AM
if you prefer, here is an outline for simple granular playback from a file:

grain.playGrain(offset, length, pitch)
skip to a offset in the file to be played back
play until length, amplitude is multiplied by a window
pitch ratio : 1 is normal, 2 is twice the pitch, 0.5 is half etc..

grain.startStream(rate, rateVariation)
begins playing grains in a continuous stream, rate defines the ms interval to trigger each grain
rateVariation sets the amount of randomness for each trigger interval

grain.Variation(offsetVariation, lengthVariation, pitchVariation)
these settings add random variation into the grain stream set in ratio 0. to 1. values
0 is no variation and 1.0 is maximum variation

grain.Settings(polyphony, window);
set the possible polyphony for how many grains can be played in the stream
set the choice for window option (hanning, blackman, hamming, etc..)

MacroMachines
01-17-2016, 02:21 PM
For my prototype 2 files simultaneously are just fine to start with. I made a little example where I can loop from the SD card and set the length and position of the loop with two encoders. It also contains the PlayLoopSdRaw I wrote about earlier. The loop length and position are both set in blocks, this was the easiest way for me to implement. The same is true for why it only reads 256 bytes from the SD card per update.

I did not check if it still works when two files are played simultaneously.

I am checking out your looper, sounds pretty good so far. I am going to see if I can make some kind of progress with it, keep me updated if you do as well! I tried playing the 4 objects with different settings for length to see what it would be like for a sort of granular and its not too bad actually, just needs a window for pop removal. maybe I can find a way to just do a quick fade in and fade out to window.

I am thinking that maybe the main play object should just be updated in the library with a "loop enable" option of some sort, as well as the start and end position settings. It only makes the object more flexible, no need for a separate loop object to just get these useful additions. I will see if I can do this sort of merger of the files and post for consideration to update library.

MacroMachines
01-17-2016, 02:27 PM
I would love to cooperate with you, I can upload all I have to Github. Do you have any of your patches available online?

I don't actually have my patches up online, but I should change that soon. I have too many really good useful ones collecting dust on my drive. I always had the mentality of "Ill post it when its done" but then I realized its never done. only took 15 years to realize it. github would be a good place so I will do a github post soon.

MacroMachines
01-17-2016, 04:05 PM
is there any way to get data into PROGMEM other then by using a utility to create a c file and copy into the code? like is it at all possible to use it in realtime to copy audio data into it?

PaulStoffregen
01-17-2016, 04:18 PM
grain.playGrain(offset, length, pitch)
....
grain.startStream(rate, rateVariation)
...
grain.Variation(offsetVariation, lengthVariation, pitchVariation)
...
grain.Settings(polyphony, window);


Any chance you could build code in C or Python (which runs on a PC) that implements these using data read from binary files and writes the 16 bit binary data to another file? It doesn't need to be efficient. The idea is to solidify the details of the algorithm, and to generate known-good test data which can be used to verify an optimized Teensy version.

MacroMachines
01-17-2016, 04:30 PM
Any chance you could build code in C or Python (which runs on a PC) that implements these using data read from binary files and writes the 16 bit binary data to another file? It doesn't need to be efficient. The idea is to solidify the details of the algorithm, and to generate known-good test data which can be used to verify an optimized Teensy version.

granular algorithm design is more of an art then science. I can certainly make / give you a pure data patch, max patch, or even simple gen~ code that does this, and I can even use the gen~ c++ compiler to give you C++ that would work somewhere. In fact if you want good c++ code and a library that you should really consider porting, check out the STK synthesis tool kit library. It is open source and has everything from nice physical models to band limited waveforms, to granular. and it is all designed to be portable and uses a generic "tick" based timing that is flexible, you just call a function to get the next sample.
https://ccrma.stanford.edu/software/stk/classstk_1_1Granulate.html

MacroMachines
01-18-2016, 03:06 AM
so this is interesting, if I set the clock to 96mhz I get cpu timing like this from the 4 loopers playing:
current , max current, max
all=2.09,73.67 Memory: 4,7
all=2.09,73.67 Memory: 4,7
all=2.09,73.67 Memory: 4,7
all=2.09,73.67 Memory: 4,7
all=15.39,73.67 Memory: 4,7
all=15.39,73.67 Memory: 4,7
all=2.09,73.67 Memory: 4,7


72MHZ
all=64.00,102.34 Memory: 4,7
all=64.00,102.34 Memory: 4,7
all=64.00,102.34 Memory: 4,7
all=64.00,102.34 Memory: 4,7
all=64.00,102.34 Memory: 4,7
all=64.00,102.34 Memory: 4,7
all=64.00,102.34 Memory: 4,7
all=35.07,102.34 Memory: 4,7
all=35.07,102.34 Memory: 4,7
all=35.07,102.34 Memory: 4,7
all=36.04,102.34 Memory: 4,7



looks like more then double the cpu usage at some points, also it seems to top out after a second of starting to play and then settle. I am playing back from SD, and astonished that 4 looping grains reading direct can perform this well. I wonder what it would be like if it were SPI flash or RAM??!

MacroMachines
01-18-2016, 03:08 AM
The idea is to solidify the details of the algorithm, and to generate known-good test data which can be used to verify an optimized Teensy version.

also if you are trying to compare the granular test data, it would be very difficult since a bit part of granular is the random variation and I am not sure how you would compare two things that are random for accuracy.

MacroMachines
01-18-2016, 03:37 AM
can you fry an sd card by writing and reading too fast? I tried having it rerecord the audio file being granulated and it didn't like something about the way I did it, and now the card doesn't seem to work. :/

MacroMachines
01-18-2016, 03:39 AM
disk utility repaired it, interesting glitch worth noting.
here is the repair info:

Verifying volume “TEENSYAUDIO”Verifying file system.** /dev/rdisk4s1
** Phase 1 - Preparing FAT
FAT[0] is incorrect (is 0x8501B4; should be 0xFFFFFF8)
Correct? no
FAT[1] is incorrect
Correct? no
** Phase 2 - Checking Directories
/: Cluster chain starting at 2 continues with cluster out of range (260702003)
Truncate? no
/POOL.RAW: Cluster chain starting at 5 continues with cluster out of range (70910920)
Truncate? no
size of /POOL.RAW is 256000, should at most be 4096
Truncate? no
/ has entries after end of directory
Truncate? no
Extend? no
/ has entries after end of directory
Truncate? no
Extend? no
/ has entries after end of directory
Truncate? no
Extend? no
/ has entries after end of directory
Truncate? no
Extend? no
/ has entries after end of directory
Truncate? no
Extend? no
/ has entries after end of directory
Truncate? no
Extend? no
/ has entries after end of directory
Truncate? no
Extend? no
/ has entries after end of directory
Truncate? no
Extend? no
** Phase 3 - Checking for Orphan Clusters
Found orphan cluster(s)
Fix? no
Found 341 orphaned clusters
Free space in FSInfo block (478480) not correct (478640)
Fix? no
5 files, 1914560 KiB free (478640 clusters)
File system check exit code is 8.Error: This disk needs to be repaired. Click Repair Disk.
Verify and Repair volume “TEENSYAUDIO”Repairing file system.** /dev/rdisk4s1
** Phase 1 - Preparing FAT
** Phase 2 - Checking Directories
** Phase 3 - Checking for Orphan Clusters
82 files, 1915216 KiB free (478804 clusters)
File system check exit code is 0.Updating boot support partitions for the volume as required.

PaulStoffregen
01-18-2016, 04:55 AM
I wonder what it would be like if it were SPI flash or RAM??!

Much better.

The SPI peripheral runs at 36 MHz when Teensy runs at at 72, but it's 48 when Teensy is at 96 MHz. Since SPI bandwidth is the limiting factor with those chips, you'll see the performance change. Likewise if you edit boards.txt to enable 120 MHz overclock!


can you fry an sd card by writing and reading too fast?

Until several months ago, I would have thought not. In theory, it should be infinitely readable, and wear leveling should spread the relatively slow write speeds you can achieve over SPI across a very large space.

But while testing the optimized SD code, I had a test sketch which played 4 short sound clips. I had it running in a loop, with all 4 overlapping, starting about 80 ms after each other. I left it running for a day, as a sort of stress test. One of the cheap Chinese cards (labeled Sandisk, but obviously a counterfeit) stopped working after many hours. That was with my optimized version, which can't ever write anything to the card.

mxxx
01-18-2016, 06:45 AM
HAHA like minded indeed.. so yeah I just spent an hour looking at the delay object code with the intent of doing just this. I would love to collaborate with you on that granular delay line object if you are willing to post or send me what you have done thus far? I have made so many max and puredata granular patches from scratch and I can confidently say I have a full understanding of all granular techniques (and I have a couple favorites that are low cpu and sound amazing). I just am learning the ropes of navigating library design and C++ Object creation.

i wasn't very persistent with the external delay code, i'm afraid. (or well, i got distracted with some other piece of hardware, with (lots) more RAM). that said, i still have the teensy codec/SRAM/SPI flash hardware, so wouldn't mind helping doing something with it.

granular player with the flash, or granular delay line with SRAM, both seems most feasible/worth exploring to me; i suspect things won't sound all that granular, but some interesting effects might result ..

MacroMachines
01-19-2016, 06:10 AM
i wasn't very persistent with the external delay code, i'm afraid. (or well, i got distracted with some other piece of hardware, with (lots) more RAM). that said, i still have the teensy codec/SRAM/SPI flash hardware, so wouldn't mind helping doing something with it.

granular player with the flash, or granular delay line with SRAM, both seems most feasible/worth exploring to me; i suspect things won't sound all that granular, but some interesting effects might result ..


I am using recter's loop playback to play 4 different looping "grains' from randomized position and loop sizes in the SD card raw file that is recorded and it sounds VERY granular. I was shocked actually. I am reworking things a bit and making my own grain playback engine which I will be showing at NAMM in a couple days :D

may I ask what your other piece of hardware with more RAM is? I want to look into that for some future projects, as RAM is my main limitation right now. Even so I am making things I never thought possible with the teensy and an SD card. perpetually quite impressed with this device and the audio library.

mxxx
01-19-2016, 07:15 AM
I am using recter's loop playback to play 4 different looping "grains' from randomized position and loop sizes in the SD card raw file that is recorded and it sounds VERY granular. I was shocked actually. I am reworking things a bit and making my own grain playback engine which I will be showing at NAMM in a couple days :D

sure, i suppose i was thinking granular as in "thousands of very short sonic grains" (http://www.jstor.org/stable/3679937?seq=1#page_scan_tab_contents) .. anyways, this should be easy enough to port to the SPI flash stuff, no? that should give you 10+ grains. the only issue with that sort of set-up i can foresee is moving the files on there in the first place (in terms of user experience), it taking forever. then again, even the 16MB ones, they'll fit 5 minutes of mono, which is plenty to play with.




may I ask what your other piece of hardware with more RAM is? I want to look into that for some future projects, as RAM is my main limitation right now. Even so I am making things I never thought possible with the teensy and an SD card. perpetually quite impressed with this device and the audio library.

sure. nothing mysterious. i've put together an owl-like thing, STM32F4 with 1MB extra SRAM; still figuring that one out. all this RAM stuff seems to be 54-pin TSSOP (at best), so not suitable for teensies, unfortunately. the other, much easier thing to do is running, say, pd on something like this (https://github.com/mxmxmx/terminal_tedium). i've recently swapped the pcm5102a for a wm8731; works pretty ok. current draw isn't too bad either (with zero, or A+), though there's fairly palpable limits, too.

MacroMachines
01-30-2016, 05:55 AM
sure, i suppose i was thinking granular as in "thousands of very short sonic grains" (http://www.jstor.org/stable/3679937?seq=1#page_scan_tab_contents) .. anyways, this should be easy enough to port to the SPI flash stuff, no? that should give you 10+ grains. the only issue with that sort of set-up i can foresee is moving the files on there in the first place (in terms of user experience), it taking forever. then again, even the 16MB ones, they'll fit 5 minutes of mono, which is plenty to play with.

Ive read Curtis Roads "Microsound", I highly recommend his "Computer Music Tutorial" book. It is a huge (literally, physically) resource with sections detailing every kind of synthesis and effect process, and how to go about coding them. The "thousands of very short sonic grains" is actually what is taking place in my test of using the 4 looper players playing very short randomized start and length loops, it equates to 4 times 20-100ms or so bits of audio per second so potentially up to a hundred or so maybe. Rarely do I ever use granular that ends up making thousands of grains per second, that is just a noise wash or chaos. It is more useful to do something like the SOGS Smooth Overlap Granular Synthesis method from IRCAM's max/msp library. That is the most amazing granular algorithm I have found in my many years of searching and testing. It is simple, clean, and allows smooth scrubbing of the position that results in a quality of time independent playback keeping the character of the original sound intact. That is quite rare to find. Ive dissected and remade the SOGS~ to a decent level in puredata and max/msp.


i've put together an owl-like thing, STM32F4 with 1MB extra SRAM; still figuring that one out. all this RAM stuff seems to be 54-pin TSSOP (at best), so not suitable for teensies, unfortunately. the other, much easier thing to do is running, say, pd on something like this (https://github.com/mxmxmx/terminal_tedium). i've recently swapped the pcm5102a for a wm8731; works pretty ok. current draw isn't too bad either (with zero, or A+), though there's fairly palpable limits, too.

RE: the owl, I would love to learn more about this and learn from/with you on that. I have a STM32F407 discovery board that is just sitting around as I had taken to the teensy for my first digital audio projects. Would that work? I looked at the OWL and I hadn't realized it was opensource, pretty cool! I just don't know where to start with moving from arduino which I code mostly in sublime text. What IDE should I get etc..

RE: Pi based audio: I tried making something like that a year and a half ago, I think around when you were doing your's, I wasn't happy with the quality/power and couldn't find any good way to get quality audio I/O outside using a usb soundcard. Were you able to connect a DAC/codec to the PI for audio I/O? I am considering getting back to the embedded linux world and evaluating the pi zero and 4d displays with built in soc. I also thought maybe using libpd to run the patches rather then puredata might somehow work better. Something about it though doesn't get me as excited as I would hope, I want something better. My dream is to create a new opensource Nord Modular G2 style platform. There are a couple cool things coming out in that realm but nothing to the depth and playful quick creative patching I want.

MacroMachines
01-30-2016, 06:18 AM
what would be the best way to load wav files from SD card into ram and then playback?

mxxx
01-30-2016, 10:58 AM
The "thousands of very short sonic grains" is actually what is taking place in my test of using the 4 looper players playing very short randomized start and length loops, it equates to 4 times 20-100ms or so bits of audio per second so potentially up to a hundred or so maybe.

a hundred or so? how would that work? isn't that "looper" basically just streaming files from SD? it won't scale much beyond that, as far as i can see. using the spi flash will work much better, i think.




I have a STM32F407 discovery board that is just sitting around as I had taken to the teensy for my first digital audio projects. Would that work? I looked at the OWL and I hadn't realized it was opensource, pretty cool! I just don't know where to start with moving from arduino which I code mostly in sublime text. What IDE should I get etc.. mostly yes, you won't have the extra SRAM and the codec is different, of course, so at the very least you'd have to adjust the codec driver stuff. other than that, sublime text will do. and gcc.



RE: Pi based audio: I tried making something like that a year and a half ago, I think around when you were doing your's, I wasn't happy with the quality/power and couldn't find any good way to get quality audio I/O outside using a usb soundcard. Were you able to connect a DAC/codec to the PI for audio I/O?

yep, there's drivers for the wm8731 ("rpi-proto") in the vanilla distro; that works/sounds pretty good and hassle-free. or you could use a fancier codec -- you've probably seen the Teensy SuperAudioBoard (https://github.com/whollender/SuperAudioBoard), it comes with everything you need for rpi, too, IIRC.

PaulStoffregen
01-30-2016, 02:39 PM
I picked up a copy of the Curtis Roads Microsound book a couple weeks ago. So far I've only read chapters 2 & 3 and skipped around through some other parts briefly. If only there were more hours in every day.....

I have been putting some thought into how I'd like to implement this in the Teensy Audio library. My main reason for reading Roads's book was to get a more general idea of the range of uses, rather than possibly going down a path with too narrow of a vision. It's also nice to get a feel for how other systems have done this.

My opinion so far is to provide a couple fairly low level but highly optimized functions in the audio library. Traditionally, the audio library objects provide a few simple Arduino style functions. But for granular synthesis (playing the grains), I've been toying with the idea of a more machine-oriented API meant for building companion libraries. Such a library would build a linked list of the grains it wants the audio library to play. You could do that yourself from an Arduino sketch, but the idea is other libraries would be made. One thing Roads's Microsound makes clear is there's a lot of different strategies for playing grains. My hope is to write an optimized low-level engine in the Teensy Audio library which does the high speed, low-level work efficiently. The actual decisions about which grains to play, when to mix each one, and what parameters each will use would happen in other libs, to be developed separately.

I understand a lot of the interest here is for live recording. There too, I'm imagining the Teensy Audio library would provide a fairly simple record function. In fact, it'll very likely be a companion to the memory player object. Decisions about how to chop up the recored waveform into grains would happen outside the audio library. You could choose regions of the recorded sound, put pointer to them into the grain specs, and tell the granular synthesis object to play them in pretty much any way you like.

Realistically, Teensy's internal RAM and Flash are probably going to be the only supported media. Perhaps in the distant future we'll expand to external SPI RAM or Flash, but there's no way I'm going to add that extra complexity and performance limit on the first attempt. So grain samples are going to be limited to about 50K of the extra RAM or about 200K of the extra internal Flash. When we get a bigger Teensy, those limits will expand to about 230K RAM and nearly a megabyte of Flash. Of course, if you have a segment of a waveform in memory, you can compose any number of grains using it.

So a big question at this early stage is what per-grain parameter should the low-level synthesis code support. So far, I have these six:

Time offset / when to begin
Play length / duration
Sample data
Window / envelope shape
Volume / gain / amplitude
Pitch stretch factor

Every extra feature supported in the low-level engine will come at the cost of extra overhead for all. So features play playing the sample data backwards would probably happen at a higher level, by simply creating a backwards copy of the samples in memory.

MacroMachines
01-30-2016, 06:00 PM
My opinion so far is to provide a couple fairly low level but highly optimized functions in the audio library. Traditionally, the audio library objects provide a few simple Arduino style functions. But for granular synthesis (playing the grains), I've been toying with the idea of a more machine-oriented API meant for building companion libraries. Such a library would build a linked list of the grains it wants the audio library to play. You could do that yourself from an Arduino sketch, but the idea is other libraries would be made. One thing Roads's Microsound makes clear is there's a lot of different strategies for playing grains. My hope is to write an optimized low-level engine in the Teensy Audio library which does the high speed, low-level work efficiently. The actual decisions about which grains to play, when to mix each one, and what parameters each will use would happen in other libs, to be developed separately.


I'm not sure if I understand fully. Are you saying you plan to create a separate library for granular? I agree with your assessment that there are many strategies, and I will look back at my previous outline of methods/properties/functions to see if I can think of any way to make them more capable. Really the most straightforward start would just be to create a single "grainPlay" object that is basically a windowed sample player that plays back a bit of a data from offset to length at a pitch ratio.

Alternately/additionally, and possibly more useful, would be to make a polyphonic voice management system in the library. A way of creating a section of a patch in the audio system tool that would be instantiated multiple times to create round-robin voice stealing as "note on" messages are sent into it, this way you could make your own grains (or synth voices) from any objects in the library and have very deliberate control over their playback. One could make grain delays by simply loading the delay object and using an envelope to window it. A granular sampler could just use the playRaw object. A "window" object that could be driven by phase could be useful.

I had thought the SD playback would be an unrealistic bottleneck to overcome but as you can see in my tests, if you set the clock to 96mhz it stabilizes at a reasonably low cpu level for 4 playback objects looping short grains (no window but sounded just fine on most source material). IF I can figure out how to buffer the start of the files it might work fantastically. I will post some video in a bit showing some examples.



I understand a lot of the interest here is for live recording. There too, I'm imagining the Teensy Audio library would provide a fairly simple record function. In fact, it'll very likely be a companion to the memory player object. Decisions about how to chop up the recored waveform into grains would happen outside the audio library. You could choose regions of the recorded sound, put pointer to them into the grain specs, and tell the granular synthesis object to play them in pretty much any way you like.
Of course, if you have a segment of a waveform in memory, you can compose any number of grains using it.


I think a buffer with read/write access would be ideal




So a big question at this early stage is what per-grain parameter should the low-level synthesis code support. So far, I have these six:

Time offset / when to begin
Play length / duration
Sample data
Window / envelope shape
Volume / gain / amplitude
Pitch stretch factor


is pitch stretch factor just pitch or something more?



Every extra feature supported in the low-level engine will come at the cost of extra overhead for all. So features play playing the sample data backwards would probably happen at a higher level, by simply creating a backwards copy of the samples in memory.

I posted a function that does this in another thread, Ill copy here:

MacroMachines
01-30-2016, 09:26 PM
a hundred or so? how would that work? isn't that "looper" basically just streaming files from SD? it won't scale much beyond that, as far as i can see. using the spi flash will work much better, i think.

Yes, this is not at all an optimal route, but I was just suprised it worked as well as it did, I really thought this route would sound horrible and not even come close to a "granular" sound, but it really suprised me:

https://www.instagram.com/p/BApXbT9hyu0/?taken-by=axiomcrux

https://www.instagram.com/p/BApXbT9hyu0/?taken-by=axiomcrux
(is it possible to embed instagram videos on here?)


mostly yes, you won't have the extra SRAM and the codec is different, of course, so at the very least you'd have to adjust the codec driver stuff. other than that, sublime text will do. and gcc.


so where would I start with the STM and the OWL thing?

john-mike
02-04-2016, 01:38 AM
Here's a granular effect I made a while ago for a forthcoming product. You can hear it used to pitch shift in the video. http://bleeplabs.com/store/thingamagoop-3000/
I made a lot of little changes and effects for the Thingamagoop3000 that I'll be cleaning up and posting when it starts shipping.

This is not too much different than what's been posted here but it works well enough for noise music!


.cpp:



#include "effect_granular.h"
#include "arm_math.h"



void AudioEffectGranular::begin(int16_t *sample_bank_def, int16_t max_len_def)
{
sample_req=1;
length(max_len_def - 1);

sample_bank=sample_bank_def;

}

void AudioEffectGranular::length(int16_t max_len_def)
{
if (max_len_def<100)
{
max_sample_len = 100;
glitch_len = max_sample_len/3;
}
else{
max_sample_len = (max_len_def - 1);
glitch_len = max_sample_len/3;
}

}


void AudioEffectGranular::freeze(int16_t activate,int16_t playpack_rate_def,int16_t freeze_length_def)
{
if (activate==1){
grain_mode = 1;
}
if (activate==0){
grain_mode = 0;
}

rate(playpack_rate_def);
if (freeze_length_def<50)
{
freeze_len=50;
}
if (freeze_length_def>=max_sample_len)
{
freeze_len=max_sample_len;
}

if (freeze_length_def>=50 && freeze_length_def<max_sample_len){
freeze_len=freeze_length_def;
}

}

void AudioEffectGranular::shift(int16_t activate,int16_t playpack_rate_def,int16_t grain_length_def)
{
if (activate==1){
grain_mode = 2;
}
if (activate==0){
grain_mode = 3;
}

rate(playpack_rate_def);
if (allow_len_change==1 )
{
// Serial.println("aL");
length(grain_length_def);

}
}


void AudioEffectGranular::rate(int16_t playpack_rate_def)
{
playpack_rate = playpack_rate_def;
}


void AudioEffectGranular::update(void)
{
audio_block_t *block;

if (sample_bank == NULL)return;

block = receiveWritable(0);

if (block) {


if (grain_mode == 3) {
//through

for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
write_head++;

if (write_head >= max_sample_len) {
write_head = 0;
}

sample_bank[write_head] = block->data[i];

}

transmit(block);

}


if (grain_mode == 0) {
//capture audio but dosen't output
//this needs to be happening at all times if you want to use grain_mode = 1, the simple "freeze" sampler.

for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
write_head++;

if (write_head >= max_sample_len) {
write_head = 0;
}

sample_bank[write_head] = block->data[i];

}
}


if (grain_mode == 1) {
//when activated the last

for (int j = 0; j < AUDIO_BLOCK_SAMPLES; j++) {

if (playpack_rate >= 0)
{
accumulator += playpack_rate;
read_head = accumulator >> 9;

}

if (read_head >= freeze_len) {
accumulator = 0;
read_head -= max_sample_len;
}

block->data[j] = sample_bank[read_head];

}
transmit(block);

}


if (grain_mode == 2) {
//GLITCH SHIFT
//basic granular synth thingy
//the shorter the sample the max_sample_len the more tonal it is. Longer it has more definition.
//its a bit roboty either way which is obv great and good enough for noise music.

for (int k = 0; k < AUDIO_BLOCK_SAMPLES; k++) {

int16_t current_input = block->data[k];

//we only want to start recodeing when the audio is crossing zero to minimize pops
if ((current_input<0 && prev_input>0))
{
zero_cross_down=1;
}

else
{
zero_cross_down=0;
}

prev_input=current_input;


if (zero_cross_down==1 && sample_req==1)
{
write_en=1;
}


if (write_en==1)
{
sample_req=0;
allow_len_change=1; //reduces noise by not allowing the length to change after the sample has been recored. kind of not too much though

if (write_head >= glitch_len) {
glitch_cross_len=glitch_len;
write_head = 0;
sample_loaded = 1;
write_en=0;
allow_len_change=0;


}

sample_bank[write_head] = block->data[k];
write_head++;

}


if (sample_loaded == 1) {
//move it to the middle third of the bank.
//3 "seperate" banks are used
float fade_len=20.00;
int16_t m2=fade_len;

for (int m = 0; m < 2; m++)
//I'm off by one somewhere? why is there a tick at the beginning of this only when it's combined with the fade out???? ooor am i osbserving that incorrectly
//either wait it works enough
{
sample_bank[m + glitch_len] = 0;
}

for (int m = 2; m < glitch_len-m2; m++)
{
sample_bank[m + glitch_len] = sample_bank[m];
}

for (int m = glitch_len-m2; m < glitch_len; m++)
//fade out the end. You can just make fadet=0 but it's a little too daleky
{
float fadet=sample_bank[m]*(m2/fade_len);
sample_bank[m + glitch_len] = (int16_t)fadet;
m2--;

}
sample_loaded = 0;
sample_req=1;
}

accumulator += playpack_rate;
read_head = (accumulator >> 9);

if (read_head >= glitch_len) {
read_head -= (glitch_len);

accumulator=0;


for (int m = 0; m < glitch_len; m++)
{
sample_bank[m + (glitch_len*2)] = sample_bank[m+glitch_len];
// sample_bank[m + (glitch_len*2)] = (m%20)*1000;
}


}


block->data[k] = sample_bank[read_head+(glitch_len*2)];

}
transmit(block);

}


release(block);
}

}



.h



#include "AudioStream.h"

class AudioEffectGranular :
public AudioStream
{
public:
AudioEffectGranular(void):
AudioStream(1,inputQueueArray) {
}

void begin(int16_t *sample_bank_def,int16_t max_len_def);
void length(int16_t max_len_def);
void rate(int16_t playpack_rate_def);

void freeze(int16_t activate,int16_t playpack_rate_def,int16_t grain_length_def);
void shift(int16_t activate,int16_t playpack_rate_def,int16_t grain_length_def);


virtual void update(void);

private:
audio_block_t *inputQueueArray[1];
int16_t *sample_bank;
int16_t max_sample_len;
int16_t write_head,read_head,grain_mode,freeze_len,allow_l en_change;
int16_t playback_rate;
int16_t capture_trigger,capture_index,current_mode,playpac k_rate;
int32_t accumulator,play_index,increment;
int16_t sample_loaded,prev_input,zero_cross_up,zero_cross_ down,sample_trigger,write_en,glitch_cross_len,load _req,glitch_len,glitch_min_len,sample_req;


};

MacroMachines
02-04-2016, 02:25 AM
Here's a granular effect I made a while ago for a forthcoming product. You can hear it used to pitch shift in the video. http://bleeplabs.com/store/thingamagoop-3000/
I made a lot of little changes and effects for the Thingamagoop3000 that I'll be cleaning up and posting when it starts shipping.

This is not too much different than what's been posted here but it works well enough for noise music!



Would you be willing to post an example sketch? I copied the two files but not entirely sure where to start with testing them, or how to use them. At least a general explanation of use?

john-mike
02-04-2016, 04:41 PM
Would you be willing to post an example sketch?

It's similar to the other time based effects. You need to define a bank and max length.



int16_t granular_bank_len = 3000;
int16_t granular_bank[3000] = {};

granular1.begin(granular_bank, granular_bank_len);


Then use one of these functions.



granular1.freeze(activate,playpack_rate,grain_leng th);

granular1.shift(activate,playpack_rate,grain_lengt h);

MacroMachines
02-05-2016, 08:34 AM
It's similar to the other time based effects. You need to define a bank and max length.


so is granular_bank an array that stores the buffer for the audio to be granulated? is activate a 1/0 boolean to turn it on/off? and I assume playback_rate is a ratio where 1 would be regular speed and 0 would be stopped, is it different in the freeze vs shift? like would shift use the playback rate to set the pitch shift ratio in that case?

john-mike
02-05-2016, 04:33 PM
so is granular_bank an array that stores the buffer for the audio to be granulated? is activate a 1/0 boolean to turn it on/off? and I assume playback_rate is a ratio where 1 would be regular speed and 0 would be stopped

Yes activate is 1/0.
Rate and length are integers but should be floats scaled like that in the final release.
Try rate is between 0-4095. Length can be 0 (though it's stops at 50) to the buffer length.



, is it different in the freeze vs shift? like would shift use the playback rate to set the pitch shift ratio in that case?

Freeze loops the last "grain_length" of incoming samples when activated.
Shift is the simple granular pitch shift. It is the same thing as freeze really but it gets a new chunk of samples when the grain_length has passed (It's not just the prev sample being done as if the rate is faster than normal speed it has to loop).

receter
02-11-2016, 08:01 AM
Haha, I love how this thread escalated. This motivates me to continue to play with my code.


My hope is to write an optimized low-level engine in the Teensy Audio library which does the high speed, low-level work efficiently. The actual decisions about which grains to play, when to mix each one, and what parameters each will use would happen in other libs, to be developed separately.


That would be awesome! Could you post a imaginary example on how this API is used? That would make it easier for me to understand the general idea and comment on it.

urbanspaceman
03-17-2018, 06:24 PM
Hi Guys
Someone of you made progress in the granular side of teensy audio lib?

jdev
05-02-2018, 08:51 PM
@urbanspaceman looks like effect_granular code is available in master:
- https://github.com/PaulStoffregen/Audio/blob/master/effect_granular.h
- https://github.com/PaulStoffregen/Audio/blob/master/effect_granular.cpp

See more at https://forum.pjrc.com/threads/41299-My-audio-h-fork-Tape-delay-FM-waveform-input-granular-effect-and-more/page2.

urbanspaceman
05-03-2018, 08:18 AM
Thank you Jdev, as soon as I have time I look