Forum Rule: Always post complete source code & details to reproduce any issue!
Page 3 of 3 FirstFirst 1 2 3
Results 51 to 63 of 63

Thread: Granular Synthesis with Teensy and Audio Adaptor

  1. #51
    Senior Member
    Join Date
    Feb 2013
    Posts
    563
    Quote Originally Posted by MacroMachines View Post
    The "thousands of very short sonic grains" is actually what is taking place in my test of using the 4 looper players playing very short randomized start and length loops, it equates to 4 times 20-100ms or so bits of audio per second so potentially up to a hundred or so maybe.
    a hundred or so? how would that work? isn't that "looper" basically just streaming files from SD? it won't scale much beyond that, as far as i can see. using the spi flash will work much better, i think.


    I have a STM32F407 discovery board that is just sitting around as I had taken to the teensy for my first digital audio projects. Would that work? I looked at the OWL and I hadn't realized it was opensource, pretty cool! I just don't know where to start with moving from arduino which I code mostly in sublime text. What IDE should I get etc..
    mostly yes, you won't have the extra SRAM and the codec is different, of course, so at the very least you'd have to adjust the codec driver stuff. other than that, sublime text will do. and gcc.

    RE: Pi based audio: I tried making something like that a year and a half ago, I think around when you were doing your's, I wasn't happy with the quality/power and couldn't find any good way to get quality audio I/O outside using a usb soundcard. Were you able to connect a DAC/codec to the PI for audio I/O?
    yep, there's drivers for the wm8731 ("rpi-proto") in the vanilla distro; that works/sounds pretty good and hassle-free. or you could use a fancier codec -- you've probably seen the Teensy SuperAudioBoard, it comes with everything you need for rpi, too, IIRC.

  2. #52
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,598
    I picked up a copy of the Curtis Roads Microsound book a couple weeks ago. So far I've only read chapters 2 & 3 and skipped around through some other parts briefly. If only there were more hours in every day.....

    I have been putting some thought into how I'd like to implement this in the Teensy Audio library. My main reason for reading Roads's book was to get a more general idea of the range of uses, rather than possibly going down a path with too narrow of a vision. It's also nice to get a feel for how other systems have done this.

    My opinion so far is to provide a couple fairly low level but highly optimized functions in the audio library. Traditionally, the audio library objects provide a few simple Arduino style functions. But for granular synthesis (playing the grains), I've been toying with the idea of a more machine-oriented API meant for building companion libraries. Such a library would build a linked list of the grains it wants the audio library to play. You could do that yourself from an Arduino sketch, but the idea is other libraries would be made. One thing Roads's Microsound makes clear is there's a lot of different strategies for playing grains. My hope is to write an optimized low-level engine in the Teensy Audio library which does the high speed, low-level work efficiently. The actual decisions about which grains to play, when to mix each one, and what parameters each will use would happen in other libs, to be developed separately.

    I understand a lot of the interest here is for live recording. There too, I'm imagining the Teensy Audio library would provide a fairly simple record function. In fact, it'll very likely be a companion to the memory player object. Decisions about how to chop up the recored waveform into grains would happen outside the audio library. You could choose regions of the recorded sound, put pointer to them into the grain specs, and tell the granular synthesis object to play them in pretty much any way you like.

    Realistically, Teensy's internal RAM and Flash are probably going to be the only supported media. Perhaps in the distant future we'll expand to external SPI RAM or Flash, but there's no way I'm going to add that extra complexity and performance limit on the first attempt. So grain samples are going to be limited to about 50K of the extra RAM or about 200K of the extra internal Flash. When we get a bigger Teensy, those limits will expand to about 230K RAM and nearly a megabyte of Flash. Of course, if you have a segment of a waveform in memory, you can compose any number of grains using it.

    So a big question at this early stage is what per-grain parameter should the low-level synthesis code support. So far, I have these six:

    Time offset / when to begin
    Play length / duration
    Sample data
    Window / envelope shape
    Volume / gain / amplitude
    Pitch stretch factor

    Every extra feature supported in the low-level engine will come at the cost of extra overhead for all. So features play playing the sample data backwards would probably happen at a higher level, by simply creating a backwards copy of the samples in memory.

  3. #53
    Senior Member
    Join Date
    Dec 2014
    Posts
    134
    Quote Originally Posted by PaulStoffregen View Post
    My opinion so far is to provide a couple fairly low level but highly optimized functions in the audio library. Traditionally, the audio library objects provide a few simple Arduino style functions. But for granular synthesis (playing the grains), I've been toying with the idea of a more machine-oriented API meant for building companion libraries. Such a library would build a linked list of the grains it wants the audio library to play. You could do that yourself from an Arduino sketch, but the idea is other libraries would be made. One thing Roads's Microsound makes clear is there's a lot of different strategies for playing grains. My hope is to write an optimized low-level engine in the Teensy Audio library which does the high speed, low-level work efficiently. The actual decisions about which grains to play, when to mix each one, and what parameters each will use would happen in other libs, to be developed separately.
    I'm not sure if I understand fully. Are you saying you plan to create a separate library for granular? I agree with your assessment that there are many strategies, and I will look back at my previous outline of methods/properties/functions to see if I can think of any way to make them more capable. Really the most straightforward start would just be to create a single "grainPlay" object that is basically a windowed sample player that plays back a bit of a data from offset to length at a pitch ratio.

    Alternately/additionally, and possibly more useful, would be to make a polyphonic voice management system in the library. A way of creating a section of a patch in the audio system tool that would be instantiated multiple times to create round-robin voice stealing as "note on" messages are sent into it, this way you could make your own grains (or synth voices) from any objects in the library and have very deliberate control over their playback. One could make grain delays by simply loading the delay object and using an envelope to window it. A granular sampler could just use the playRaw object. A "window" object that could be driven by phase could be useful.

    I had thought the SD playback would be an unrealistic bottleneck to overcome but as you can see in my tests, if you set the clock to 96mhz it stabilizes at a reasonably low cpu level for 4 playback objects looping short grains (no window but sounded just fine on most source material). IF I can figure out how to buffer the start of the files it might work fantastically. I will post some video in a bit showing some examples.

    Quote Originally Posted by PaulStoffregen View Post
    I understand a lot of the interest here is for live recording. There too, I'm imagining the Teensy Audio library would provide a fairly simple record function. In fact, it'll very likely be a companion to the memory player object. Decisions about how to chop up the recored waveform into grains would happen outside the audio library. You could choose regions of the recorded sound, put pointer to them into the grain specs, and tell the granular synthesis object to play them in pretty much any way you like.
    Of course, if you have a segment of a waveform in memory, you can compose any number of grains using it.
    I think a buffer with read/write access would be ideal


    Quote Originally Posted by PaulStoffregen View Post
    So a big question at this early stage is what per-grain parameter should the low-level synthesis code support. So far, I have these six:

    Time offset / when to begin
    Play length / duration
    Sample data
    Window / envelope shape
    Volume / gain / amplitude
    Pitch stretch factor
    is pitch stretch factor just pitch or something more?

    Quote Originally Posted by PaulStoffregen View Post
    Every extra feature supported in the low-level engine will come at the cost of extra overhead for all. So features play playing the sample data backwards would probably happen at a higher level, by simply creating a backwards copy of the samples in memory.
    I posted a function that does this in another thread, Ill copy here:

  4. #54
    Senior Member
    Join Date
    Dec 2014
    Posts
    134
    Quote Originally Posted by mxxx View Post
    a hundred or so? how would that work? isn't that "looper" basically just streaming files from SD? it won't scale much beyond that, as far as i can see. using the spi flash will work much better, i think.
    Yes, this is not at all an optimal route, but I was just suprised it worked as well as it did, I really thought this route would sound horrible and not even come close to a "granular" sound, but it really suprised me:

    https://www.instagram.com/p/BApXbT9h...n-by=axiomcrux
    https://www.instagram.com/p/BApXbT9h...n-by=axiomcrux
    (is it possible to embed instagram videos on here?)

    Quote Originally Posted by mxxx View Post
    mostly yes, you won't have the extra SRAM and the codec is different, of course, so at the very least you'd have to adjust the codec driver stuff. other than that, sublime text will do. and gcc.
    so where would I start with the STM and the OWL thing?

  5. #55
    Member
    Join Date
    Mar 2013
    Location
    Austin, TX
    Posts
    92
    Here's a granular effect I made a while ago for a forthcoming product. You can hear it used to pitch shift in the video. http://bleeplabs.com/store/thingamagoop-3000/
    I made a lot of little changes and effects for the Thingamagoop3000 that I'll be cleaning up and posting when it starts shipping.

    This is not too much different than what's been posted here but it works well enough for noise music!


    .cpp:
    Code:
    #include "effect_granular.h"
    #include "arm_math.h"
    
    
    
    void AudioEffectGranular::begin(int16_t *sample_bank_def, int16_t max_len_def)
    {
      sample_req=1;
      length(max_len_def - 1);
    
      sample_bank=sample_bank_def;
    
    }
    
    void AudioEffectGranular::length(int16_t max_len_def)
    {
      if (max_len_def<100)
      {
      max_sample_len = 100;
      glitch_len = max_sample_len/3;
      }
      else{
      max_sample_len = (max_len_def - 1);
      glitch_len = max_sample_len/3;
      }
    
      }
    
    
    void AudioEffectGranular::freeze(int16_t activate,int16_t playpack_rate_def,int16_t freeze_length_def)
    {
      if (activate==1){
      grain_mode = 1;
      }
      if (activate==0){
      grain_mode = 0;
      }
    
      rate(playpack_rate_def);
      if (freeze_length_def<50)
      {
        freeze_len=50;
      }
      if (freeze_length_def>=max_sample_len)
      {
        freeze_len=max_sample_len;
      }
    
      if (freeze_length_def>=50 && freeze_length_def<max_sample_len){
        freeze_len=freeze_length_def;
      }
    
    }
    
    void AudioEffectGranular::shift(int16_t activate,int16_t playpack_rate_def,int16_t grain_length_def)
    {
      if (activate==1){
      grain_mode = 2;
      }
      if (activate==0){
      grain_mode = 3;
      }
    
      rate(playpack_rate_def);
      if (allow_len_change==1 )
      {
      //  Serial.println("aL");
          length(grain_length_def);
    
      }
    }
    
    
    void AudioEffectGranular::rate(int16_t playpack_rate_def)
    {
      playpack_rate = playpack_rate_def;
    }
    
    
    void AudioEffectGranular::update(void)
    {
      audio_block_t *block;
    
      if (sample_bank == NULL)return;
    
      block = receiveWritable(0);
    
      if (block) {
    
    
        if (grain_mode == 3) {
          //through
    
          for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
            write_head++;
    
            if (write_head >= max_sample_len) {
              write_head = 0;
            }
    
            sample_bank[write_head] = block->data[i];
    
          }
    
          transmit(block);
    
        }
    
    
        if (grain_mode == 0) {
          //capture audio but dosen't output
          //this needs to be happening at all times if you want to use grain_mode = 1, the simple "freeze" sampler.
    
          for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
            write_head++;
    
            if (write_head >= max_sample_len) {
              write_head = 0;
            }
    
            sample_bank[write_head] = block->data[i];
    
          }
        }
    
    
        if (grain_mode == 1) {
        //when activated the last 
    
          for (int j = 0; j < AUDIO_BLOCK_SAMPLES; j++) {
    
            if (playpack_rate >= 0)
            {
              accumulator += playpack_rate;
              read_head = accumulator >> 9;
    
            }
    
            if (read_head >= freeze_len) {
              accumulator = 0;
              read_head -= max_sample_len;
            }
    
            block->data[j] = sample_bank[read_head];
    
          }
          transmit(block);
    
        }
    
    
        if (grain_mode == 2) { 
          //GLITCH SHIFT
          //basic granular synth thingy 
          //the shorter the sample the max_sample_len the more tonal it is. Longer it has more definition. 
          //its a bit roboty either way which is obv great and good enough for noise music.
    
          for (int k = 0; k < AUDIO_BLOCK_SAMPLES; k++) {
    
            int16_t current_input = block->data[k];
    
            //we only want to start recodeing when the audio is crossing zero to minimize pops
            if ((current_input<0 && prev_input>0))
            {
              zero_cross_down=1;
            }
    
            else
            {
              zero_cross_down=0;
            }
    
            prev_input=current_input;
    
    
            if (zero_cross_down==1 && sample_req==1)
            {
              write_en=1;
            }
    
    
            if (write_en==1)
            {
                sample_req=0;
                allow_len_change=1; //reduces noise by not allowing the length to change after the sample has been recored. kind of not too much though 
    
              if (write_head >= glitch_len) {
                glitch_cross_len=glitch_len;
                write_head = 0;
                sample_loaded = 1;
                write_en=0;
                allow_len_change=0; 
    
    
              }
    
              sample_bank[write_head] = block->data[k];
              write_head++;
    
            }
    
    
            if (sample_loaded == 1) {
              //move it to the middle third of the bank. 
              //3 "seperate" banks are used
              float fade_len=20.00;
              int16_t m2=fade_len;
    
              for (int m = 0; m < 2; m++)  
              //I'm off by one somewhere? why is there a tick at the beginning of this only when it's combined with the fade out???? ooor am i osbserving that incorrectly
              //either wait it works enough
              {
                sample_bank[m + glitch_len] = 0;    
              }
    
              for (int m = 2; m < glitch_len-m2; m++)
              {
                sample_bank[m + glitch_len] = sample_bank[m];
              }
    
              for (int m = glitch_len-m2; m < glitch_len; m++)
                //fade out the end. You can just make fadet=0 but it's a little too daleky 
              {
                float fadet=sample_bank[m]*(m2/fade_len);
                sample_bank[m + glitch_len] = (int16_t)fadet;
                m2--;
    
              }
              sample_loaded = 0;
              sample_req=1;
            }
    
            accumulator += playpack_rate;
            read_head = (accumulator >> 9);
      
            if (read_head >= glitch_len) {
              read_head -= (glitch_len);
    
              accumulator=0;
    
    
              for (int m = 0; m < glitch_len; m++)
              {
                sample_bank[m + (glitch_len*2)] = sample_bank[m+glitch_len];
              //  sample_bank[m + (glitch_len*2)] = (m%20)*1000;
              }
    
    
            }
    
    
            block->data[k] = sample_bank[read_head+(glitch_len*2)];
    
          }
          transmit(block);
    
        }
    
    
        release(block);
      }
    
    }
    .h
    Code:
    #include "AudioStream.h"
    
    class AudioEffectGranular : 
    public AudioStream
    {
    public:
      AudioEffectGranular(void): 
      AudioStream(1,inputQueueArray) { 
      }
    
      void begin(int16_t *sample_bank_def,int16_t max_len_def);
      void length(int16_t max_len_def);
      void rate(int16_t playpack_rate_def);
    
      void freeze(int16_t activate,int16_t playpack_rate_def,int16_t grain_length_def);
      void shift(int16_t activate,int16_t playpack_rate_def,int16_t grain_length_def);
    
    
      virtual void update(void);
      
    private:
      audio_block_t *inputQueueArray[1];
      int16_t *sample_bank;
      int16_t max_sample_len;
      int16_t write_head,read_head,grain_mode,freeze_len,allow_len_change;
      int16_t playback_rate;
      int16_t capture_trigger,capture_index,current_mode,playpack_rate;
      int32_t accumulator,play_index,increment;
      int16_t sample_loaded,prev_input,zero_cross_up,zero_cross_down,sample_trigger,write_en,glitch_cross_len,load_req,glitch_len,glitch_min_len,sample_req;
    
    
    };

  6. #56
    Senior Member
    Join Date
    Dec 2014
    Posts
    134
    Quote Originally Posted by john-mike View Post
    Here's a granular effect I made a while ago for a forthcoming product. You can hear it used to pitch shift in the video. http://bleeplabs.com/store/thingamagoop-3000/
    I made a lot of little changes and effects for the Thingamagoop3000 that I'll be cleaning up and posting when it starts shipping.

    This is not too much different than what's been posted here but it works well enough for noise music!
    Would you be willing to post an example sketch? I copied the two files but not entirely sure where to start with testing them, or how to use them. At least a general explanation of use?
    Last edited by MacroMachines; 02-04-2016 at 03:08 AM.

  7. #57
    Member
    Join Date
    Mar 2013
    Location
    Austin, TX
    Posts
    92
    Quote Originally Posted by MacroMachines View Post
    Would you be willing to post an example sketch?
    It's similar to the other time based effects. You need to define a bank and max length.

    Code:
      int16_t granular_bank_len = 3000;
      int16_t granular_bank[3000] = {};
    
      granular1.begin(granular_bank, granular_bank_len);
    Then use one of these functions.

    Code:
          granular1.freeze(activate,playpack_rate,grain_length);
    
          granular1.shift(activate,playpack_rate,grain_length);

  8. #58
    Senior Member
    Join Date
    Dec 2014
    Posts
    134
    Quote Originally Posted by john-mike View Post
    It's similar to the other time based effects. You need to define a bank and max length.
    so is granular_bank an array that stores the buffer for the audio to be granulated? is activate a 1/0 boolean to turn it on/off? and I assume playback_rate is a ratio where 1 would be regular speed and 0 would be stopped, is it different in the freeze vs shift? like would shift use the playback rate to set the pitch shift ratio in that case?

  9. #59
    Member
    Join Date
    Mar 2013
    Location
    Austin, TX
    Posts
    92
    Quote Originally Posted by MacroMachines View Post
    so is granular_bank an array that stores the buffer for the audio to be granulated? is activate a 1/0 boolean to turn it on/off? and I assume playback_rate is a ratio where 1 would be regular speed and 0 would be stopped
    Yes activate is 1/0.
    Rate and length are integers but should be floats scaled like that in the final release.
    Try rate is between 0-4095. Length can be 0 (though it's stops at 50) to the buffer length.


    , is it different in the freeze vs shift? like would shift use the playback rate to set the pitch shift ratio in that case?
    Freeze loops the last "grain_length" of incoming samples when activated.
    Shift is the simple granular pitch shift. It is the same thing as freeze really but it gets a new chunk of samples when the grain_length has passed (It's not just the prev sample being done as if the rate is faster than normal speed it has to loop).

  10. #60
    Junior Member
    Join Date
    Jul 2015
    Posts
    12
    Haha, I love how this thread escalated. This motivates me to continue to play with my code.

    Quote Originally Posted by PaulStoffregen View Post
    My hope is to write an optimized low-level engine in the Teensy Audio library which does the high speed, low-level work efficiently. The actual decisions about which grains to play, when to mix each one, and what parameters each will use would happen in other libs, to be developed separately.
    That would be awesome! Could you post a imaginary example on how this API is used? That would make it easier for me to understand the general idea and comment on it.

  11. #61
    Senior Member
    Join Date
    Nov 2015
    Posts
    174
    Hi Guys
    Someone of you made progress in the granular side of teensy audio lib?

  12. #62

  13. #63
    Senior Member
    Join Date
    Nov 2015
    Posts
    174
    Thank you Jdev, as soon as I have time I look

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •