Granular Synthesis with Teensy and Audio Adaptor

Status
Not open for further replies.

receter

Member
I want to build a granular synth that I can control via MIDI. First I wanted to build it with an Arduino DUE, but then I found the Teensy. It got my attention because of the 16bit ADC.

I already bought a Teensy3.1 and the Audio Adaptor Board. Are there already some granular synth projects for Teensy you know of? I tried google but did not find any. I was just wondering because it seems to me that this is the perfect board for it.

Thanks!
 
I've not heard of anyone doing this yet. I hope you'll share your progress here.

The Teensy Audio Library processes audio in 128 sample blocks, which is approx 3 ms at 44.1 kHz sample rate. If your grain size is a multiple of 3 ms, this should be pretty achievable. If you're looking to get down to under 3 ms timing, you'll have a lot of really tough programming to do!
 
Yea, I will totally share my progress and code here. 3ms should be fairly enough I think. The Audio System Design Tool looks pretty handy, I like how every item has such a detailed explanation with photos and circuitry. I guess the audio input would be as easy as connecting a adc to a queue and write some loop that writes the queue content to memory, isn't it?

I am already looking forward to get my boards next week.
 
Here is something I quickly tried out which copies the incoming audio to a buffer and plays it back with the arbitrary waveform generator. In effect it currently lets you crudely pitch shift the audio with the built in trimpot that can be installed on the audio board. I started to experiment with using a sine wave to envelope two different arbitrary waveform generators, but then I filed it away for later use. The two sine wave windows are 180 degrees out of phase and triggered each time the buffer fills(obviously this could and should be played with). If you come up with any advancement from this please share as I am curious what could be possible.

Code:
// Nicholas C. Raftis III arbitrary waveform pitch shifting/granular test v0.01
// use the optional trimpot on the audio board to adjust playback pitch of wavetables
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>

AudioInputI2S            i2s1;           //xy=57,61
AudioRecordQueue         queue1;         //xy=176,60
AudioSynthWaveformSine   sine1;          //xy=305,224.75
AudioSynthWaveformDc     dc1;            //xy=305,258.75
AudioSynthWaveform       waveform1;      //xy=451,186.75
AudioMixer4              mixer1;         //xy=456,239.75
AudioEffectMultiply      multiply1;      //xy=599,202.75
AudioSynthWaveformSine   sine2;          //xy=691.0000095367432,340.0000066757202
AudioSynthWaveformDc     dc2;            //xy=692.2500095367432,376.2500066757202
AudioSynthWaveform       waveform2;      //xy=843.5000057220459,298.7500009536743
AudioMixer4              mixer2;         //xy=853.5000133514404,368.7500066757202
AudioEffectMultiply      multiply2;      //xy=998.5000133514404,337.5000066757202
AudioMixer4              mixer3;         //xy=1132.2500114440918,192.5
AudioOutputI2S           i2s2;           //xy=1257.75,192.74999618530273
AudioConnection          patchCord1(i2s1, 0, queue1, 0);
AudioConnection          patchCord2(sine1, 0, mixer1, 0);
AudioConnection          patchCord3(dc1, 0, mixer1, 1);
AudioConnection          patchCord4(waveform1, 0, multiply1, 0);
AudioConnection          patchCord5(mixer1, 0, multiply1, 1);
AudioConnection          patchCord6(multiply1, 0, mixer3, 0);
AudioConnection          patchCord7(sine2, 0, mixer2, 0);
AudioConnection          patchCord8(dc2, 0, mixer2, 1);
AudioConnection          patchCord9(waveform2, 0, multiply2, 0);
AudioConnection          patchCord10(mixer2, 0, multiply2, 1);
AudioConnection          patchCord11(multiply2, 0, mixer3, 1);
AudioConnection          patchCord12(mixer3, 0, i2s2, 0);
AudioControlSGTL5000     sgtl5000_1;     //xy=1415.75,474.75


int16_t buffer1[256];
int16_t buffer2[256];

int tableFreq = 172;


void setup(){
  
  AudioMemory(64);
  
  sgtl5000_1.enable();
  sgtl5000_1.inputSelect(AUDIO_INPUT_LINEIN);
  sgtl5000_1.volume(0.8);
  sgtl5000_1.lineInLevel(0);
  sgtl5000_1.lineOutLevel(13);
  
  waveform1.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
  waveform1.arbitraryWaveform(buffer1, 344);
  waveform2.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
  waveform2.arbitraryWaveform(buffer2, 344);

  sine1.amplitude(0.5);
  sine2.amplitude(0.5);
  dc1.amplitude(0.5);
  dc2.amplitude(0.5);
  
  queue1.begin();

}

void loop(){
  tableFreq = map(analogRead(15), 0, 1024, 172/2, 172*2);
  
  if(queue1.available() >= 2){ // input buffer
    memcpy(buffer1, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    memcpy(buffer2, queue1.readBuffer(), 256);
    queue1.freeBuffer();

    AudioNoInterrupts();
    waveform1.phase(0);
    waveform1.frequency(tableFreq);
    sine1.phase(0);
    sine1.frequency(tableFreq);
      
    waveform2.phase(180);
    waveform2.frequency(tableFreq);
    sine2.phase(180);
    sine2.frequency(tableFreq);
    AudioInterrupts(); }
}
 
Last edited:
Another slight variation that has a kindof interesting digital distortion effect


Code:
// Nicholas C. Raftis III granular teensy test v0.04

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>

// GUItool: begin automatically generated code
AudioInputI2S            i2s1;           //xy=55,38
AudioRecordQueue         queue2;         //xy=186,38
AudioSynthWaveformSine   sine1;          //xy=302.85713958740234,79.99999904632568
AudioSynthWaveform       waveform1;      //xy=312.2856979370117,40.42856788635254
AudioEffectMultiply      multiply1;      //xy=455.7142906188965,45.71428680419922
AudioSynthWaveformSine   sine2;          //xy=308.57141494750977,162.85712099075317
AudioSynthWaveform       waveform2;      //xy=308.5714416503906,128.5714235305786
AudioEffectMultiply      multiply2;      //xy=452.8571363176618,132.85714135851177
AudioSynthWaveformSine   sine3;          //xy=310.00000762939453,270.0000009536743
AudioSynthWaveform       waveform3;      //xy=304.2857131958008,231.4285593032837
AudioEffectMultiply      multiply3;      //xy=451.4286422729492,245.71429538726807
AudioSynthWaveformSine   sine4;          //xy=297.1428527832031,361.42855644226074
AudioSynthWaveform       waveform4;      //xy=302.8571472167969,321.428564786911
AudioEffectMultiply      multiply4;      //xy=467.14283752441406,329.9999837875366
AudioMixer4              mixer1;         //xy=628.5714378356934,187.14285564422607
AudioPlayQueue           queue1;         //xy=720.4285888671875,361.2857074737549
AudioOutputI2S           i2s2;           //xy=757.4286193847656,189.71427631378174
AudioConnection          patchCord1(i2s1, 0, queue2, 0);
AudioConnection          patchCord2(sine4, 0, multiply4, 1);
AudioConnection          patchCord3(sine1, 0, multiply1, 1);
AudioConnection          patchCord4(waveform4, 0, multiply4, 0);
AudioConnection          patchCord5(waveform3, 0, multiply3, 0);
AudioConnection          patchCord6(waveform2, 0, multiply2, 0);
AudioConnection          patchCord7(sine2, 0, multiply2, 1);
AudioConnection          patchCord8(sine3, 0, multiply3, 1);
AudioConnection          patchCord9(waveform1, 0, multiply1, 0);
AudioConnection          patchCord10(multiply3, 0, mixer1, 2);
AudioConnection          patchCord11(multiply2, 0, mixer1, 1);
AudioConnection          patchCord12(multiply1, 0, mixer1, 0);
AudioConnection          patchCord13(multiply4, 0, mixer1, 3);
AudioConnection          patchCord14(mixer1, 0, i2s2, 0);
AudioControlSGTL5000     sgtl5000_1;     //xy=744.5713729858398,254.42857027053833
// GUItool: end automatically generated code

int16_t buffer[1024];
int tableFreq = 172;


void setup(){
  AudioMemory(64);
  sgtl5000_1.enable();
  sgtl5000_1.inputSelect(AUDIO_INPUT_LINEIN);
  sgtl5000_1.volume(0.8);
  sgtl5000_1.lineInLevel(0);
  sgtl5000_1.lineOutLevel(13);
  queue2.begin();
  waveform1.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
  waveform1.arbitraryWaveform(buffer, 344);
  waveform2.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
  waveform2.arbitraryWaveform(buffer+256, 344);
  waveform3.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
  waveform3.arbitraryWaveform(buffer+512, 344);
  waveform4.begin(0.8, tableFreq, WAVEFORM_ARBITRARY);
  waveform4.arbitraryWaveform(buffer+768, 344);
  sine1.amplitude(0.8);
  sine2.amplitude(0.8);
  sine3.amplitude(0.8);
  sine4.amplitude(0.8);
}

void loop(){
  tableFreq = map(analogRead(15), 0, 1024, 172, 344);
  
  if(queue2.available() >= 4){ // input buffer
    memcpy(buffer, queue2.readBuffer(), 256);
    AudioNoInterrupts();
    queue2.freeBuffer();
    waveform1.phase(0);
    waveform1.frequency(tableFreq);
    sine1.phase(0);
    sine1.frequency(tableFreq);
    AudioInterrupts();
    
    memcpy(buffer+256, queue2.readBuffer(), 256);
    AudioNoInterrupts();
    queue2.freeBuffer();
    waveform2.phase(0);
    waveform2.frequency(tableFreq);
    sine2.phase(0);
    sine2.frequency(tableFreq);
    AudioInterrupts();
    
    memcpy(buffer+512, queue2.readBuffer(), 256);
    AudioNoInterrupts();
    queue2.freeBuffer();
    waveform3.phase(0);
    waveform3.frequency(tableFreq);
    sine3.phase(0);
    sine3.frequency(tableFreq);
    AudioInterrupts();
    
    memcpy(buffer+768, queue2.readBuffer(), 256);
    AudioNoInterrupts();
    queue2.freeBuffer();
    waveform4.phase(0);
    waveform4.frequency(tableFreq);
    sine4.phase(0);
    sine4.frequency(tableFreq);
    AudioInterrupts();

  }
}
 
Thanks! I'm sure this will help me getting started. I love it to have some code as a starting point.

My boards just arrived this morning, I will play around with your code this week and keep you posted what I have come up with.
 
I should mention that they may not sound very much like granular in their current form, the main thing that would need to happen to get a more granular type effect would be to change the timings of the windows/retriggering for each grain, maybe add some randomness to that timing, and possibly find a better and more consolidated way to play the individual grains.. possibly a grain function or even a new audio object entirely
 
Also worth looking into is the MOZZI library, I really would love to adapt some of their objects into the teensy audio library.
 
Thanks, I will definitely take a look into MOZZI later, looks quite comprehensive. I already have your code up and running, sounds great :) The next thing I want to try is to make an envelope for the grains. Then I want to build a button where I can record some seconds of audio. The grains will then be picked from this recorded audio sample in different ways. For the beginning I will add an encoder that can select the origin of the grains content in the sample. A second encoder could control the frequency in which the grains are repeated. And a third could control the length of the grain. That would be already a nice thing to have!
 
Are you trying to make it sample an incoming stream?
Also it would be great to detect zero crossings so there is no useless aliasing, it might sound better than to use fade in and out on the window edges. I havent looked at the code yet, but im no code god.
 
Resampling should prove useful if it would be musical, like if you choose a note, the selected grain would play back at that rate.
 
For those wanting more examples of granular synthesis in a finished (and IMHO rad) product:

http://www.bastl-instruments.com/instruments/microgranny/

Oh look, there is well commented code! Not the latest version with all the 2.4 features, but still...readable, commented code:

https://github.com/bastl-instruments/microGranny2

Notable comments:

* based on WaveRP library Adafruit Wave Shield - Copyright (C) 2009 by William Greiman
* -library heavily hacked - BIG THX https://code.google.com/p/waverp/

* -thanks for understanding basics thru Mozzi library http://sensorium.github.io/Mozzi/
* -written in Arduino + using SDFat library
 
@MacroMachines
While reading your code I noticed this part:

Code:
    memcpy(buffer1, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    memcpy(buffer2, queue1.readBuffer(), 256);
    queue1.freeBuffer();

This copies the output of queue1.readBuffer() to the buffer which holds the arbitrary waveform. As I understand it the arbitrary waveform needs 256 samples = 512bytes, so this fills only half of the arbitrary waveform. Shouldn't it be more like this, or do I miss something?

Code:
if(queue1.available() >= 4){ // input buffer

    memcpy(buffer1, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    memcpy(buffer1+256, queue1.readBuffer(), 256);
    queue1.freeBuffer();

    memcpy(buffer2, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    memcpy(buffer2+256, queue1.readBuffer(), 256);
    queue1.freeBuffer();

}

@drjohn
Thanks, I already stumbled upon this project but I did not take a closer look at it yet. Do you know if it it supports overlapping grains?

@nagual
I want to be able to record some seconds of audio input to memory and take this as the origin for the grain content. Zero crossing is sure a good thing to avoid clipping without the need of an envelope.
 
I think I posted 2 versions. I believe I did do what you mention in one of my versions, this was an hour worth of toying around total so I haven't gotten very far as of yet. I have big plans for this project after my current product is finished. I have already added some new useful functionality into the playSDraw object, and plan to extend the wavetable and sound file objects as well while I make progress on my new product, and will share my results openly. I have studied and created a few detailed granular synthesis engines in puredata and maxmsp/gen~ and I know pretty well how it all works in theory if you want any help or pseudo code.

@MacroMachines
While reading your code I noticed this part:

Code:
    memcpy(buffer1, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    memcpy(buffer2, queue1.readBuffer(), 256);
    queue1.freeBuffer();

This copies the output of queue1.readBuffer() to the buffer which holds the arbitrary waveform. As I understand it the arbitrary waveform needs 256 samples = 512bytes, so this fills only half of the arbitrary waveform. Shouldn't it be more like this, or do I miss something?

Code:
if(queue1.available() >= 4){ // input buffer

    memcpy(buffer1, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    memcpy(buffer1+256, queue1.readBuffer(), 256);
    queue1.freeBuffer();

    memcpy(buffer2, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    memcpy(buffer2+256, queue1.readBuffer(), 256);
    queue1.freeBuffer();

}

@Paul I love the teensy so much, thank you for this beautiful tool! I am checking out Mozzi and so far it seems to have a few more creative synthesis tools, whereas teensy audio lib has more utilitarian functions. I also notice that much of the input variables in teensy audio lib functions are float based, where Mozzi is all fixed point based. I was shocked how good the PWM output sounds right out the gate. Ideally I plan to very soon study both libraries in depth and likely port some of the useful stuff over to teensy audio lib. Off topic: I also have made my current product splitting the programmer from the Freescale and core and it works great!
 
What is the best way too loop audio data with the Teensy Audio Library?

I found this 3 Methods:

AudioPlayMemory
I guess I can pass an array of 16bit samples to the play() function and it starts playing. What want to know is how to immediately play it again when it reaches the end.
I could check for isPlaying() in the loop() but this wouldn't be 100% perfectly on time right? Is there some interrupt or something that is called after lets say 128 samples were played?
Further I assume that the data has to be a multiple of 128 samples as the library processes audio in 128 sample blocks?

AudioPlayQueue
Looks like the most efficient way, but how can I add more than 128 samples to the queue? How do I know when it has been played?

AudioSynthWaveform / WAVEFORM_ARBITRARY
This is how @MacroMachines did it but it gets quite complex if longer waveforms/loops are needed. For every 6ms I need to create another AudioSynthWaveform with an envelope and mix all together.

I took all my information from the Audio Design Tool, is there another place where the Audio Library is documented in more detail?
 
That sounds great, looking forward to it. I also read a lot about granular synthesis and I'm so excited to put together something that works.

It's the same in both versions, I think you may have accidentally mixed samples with bytes. Your buffer array from the second snippet (int16_t buffer[1024]; ) has 2048bytes and there are four 256 pointer increments were you write 256 bytes which means 1024 bytes are filled.
 
Last edited:
@MacroMachines

My code was wrong, I think it needs to be like this:

Pointer increments are 16bit as it is a 16bit type. The value for memcpy is in bytes which results in 256.

Code:
      memcpy(buffer1, queue1.readBuffer(), 256);
      queue1.freeBuffer();
      memcpy(buffer1+128, queue1.readBuffer(), 256);
      queue1.freeBuffer();
      memcpy(buffer2, queue1.readBuffer(), 256);
      queue1.freeBuffer();
      memcpy(buffer2+128, queue1.readBuffer(), 256);
      queue1.freeBuffer();
 
What is the best way too loop audio data with the Teensy Audio Library?

I found this 3 Methods:

[...]

i suppose/hope the recent SPIRAM / "memoryboard" add-ons should be useful for this kind of stuff.

i started playing around a bit with effect_delay_ext, trying to turn it into a granulator object; will have to see how that goes. it'll require lots of access per update, obviously, even more when interpolating the samples.
 
My recent experiments are with PlaySdRaw. I bought a microSDHC card and modified the PlaySdRaw class to loop the data. I now have a PlayLoopSdRaw class where I can loop a sample from the SD card and set the length and position that should be looped in the raw file.

It does not have an envelope, anti aliasing or zero-detection yet but it sounds good already. I will post the class later or tomorrow.

I will run some tests on how many loops can be played and mixed simultaneously. How much faster is the SPIRAM / "memoryboard"?
 
How much faster is the SPIRAM / "memoryboard"?

i don't have any numbers re speed (probably Paul or Frank will be able to tell you more). but as a buffer thing those ICs have a number of advantages vs microsd. random access, obviously; no 512 byte sectors, no delay when writing.
 
How much faster is the SPIRAM / "memoryboard"?

It's dramatically faster than SD cards.... and that's without much SPI FIFO optimization in the code yet!

If you're playing more than 2 files simultaneously, or even 2 from a low quality SD card, you'll have much better luck with the SPI flash & ram chips.

Testing so far is looking like about 11 to 15 simultaneous mono streams can be supported. If you use the delay line object, feeding data into the RAM counts as 1 stream, and each delay tap counts as another stream. If you're just playing raw data from SPI flash, that counts as 1 stream. Of course, if you create several raw player objects reading from the flash chip, each counts as 1 stream.

SD cards are terribly slow. Nearly all cards have very substantial access latency, at least in SPI mode. From the moment the SD library asks the card for data until the moment the card returns the "I'm ready" token is often almost as long as reading the actual 512 bytes of data, which makes SD card data transfer effectively half the speed it could have been, if the card didn't spend so much time processing commands.Add on top of that FAT filesystem overhead. At the end of each allocation cluster, the SD library needs to read another sector from the FAT. That doesn't happen very often, but when it does, two 512 byte sectors need to be read.

For glitchless audio, the worst case access time is what matters. The SPI flash have fast perfectly consistent read access times. So do the RAM chips for read and write. The time taken is *always* the same, a fixed (and short) number of SPI clock cycles. Compare that with SD cards, where you have to send a request and then the card is busy for some unknown (and lengthy) amount of time.

My hope is the SD card latency will improve when we have a future Teensy with 4-bit SDIO mode. Such a future Teensy will also have more RAM, so a future version of the SD library might use more RAM for caching. I've been told multi-sector reads also suffer only short latency between sectors, so when we have more RAM for caching, maybe a future version of the library might allocate larger than 1 sector caching blocks (which will eat up quite a bit of RAM).

But for SPI mode access, those little flash and ram chips are vastly superior for audio.
 
That sounds great it's exactly what I will need! What is the "memoryboard"? Ist it some sort of shield for the teensy? The SPIRAM is a chip like this I guess: link? Do you have any recommendations?

For my prototype 2 files simultaneously are just fine to start with. I made a little example where I can loop from the SD card and set the length and position of the loop with two encoders. It also contains the PlayLoopSdRaw I wrote about earlier. The loop length and position are both set in blocks, this was the easiest way for me to implement. The same is true for why it only reads 256 bytes from the SD card per update.

I did not check if it still works when two files are played simultaneously.
 

Attachments

  • teensy_granular_3.zip
    5.8 KB · Views: 226
Of course, if you're using only short grains, perhaps a short delay using the Teensy's memory could be enough? You can get up to about 0.5 second with the built-in memory.
 
Status
Not open for further replies.
Back
Top