Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 11 of 11

Thread: High level question about block processing in Audio library

  1. #1
    Junior Member
    Join Date
    Oct 2021
    Location
    Scotland
    Posts
    7

    High level question about block processing in Audio library

    Hi there,

    I'd like to understand how block processing happens at a high level when the audio "graph" contains loops, as in the case below (waveform2 gets fm from waveform1, waveform1 gets pwm from waveform2). Given that the Audio library processes AUDIO_BLOCK_SAMPLES at a time, I see the following options:

    1. always use data that lags by AUDIO_BLOCK_SAMPLES (each waveform uses the "cached" result of the previous block processing)
    2. arbitrarily (or otherwise by some set of rules), break the cycle and process (say) waveform1 then use the result for waveform2
    3. something else?


    If it's 2. are there some rules I might understand?




    Code:
    #include <Audio.h>
    #include <Wire.h>
    #include <SPI.h>
    #include <SD.h>
    #include <SerialFlash.h>
    
    // GUItool: begin automatically generated code
    AudioSynthWaveformModulated waveformMod2;   //xy=491.20001220703125,327
    AudioSynthWaveformModulated waveformMod1;   //xy=495.20001220703125,416
    AudioOutputUSB           usb1;           //xy=772.2000122070312,391
    AudioOutputI2S           i2s1;           //xy=772.2000122070312,444
    AudioConnection          patchCord1(waveformMod2, 0, waveformMod1, 0);
    AudioConnection          patchCord2(waveformMod2, 0, usb1, 0);
    AudioConnection          patchCord3(waveformMod1, 0, usb1, 1);
    AudioConnection          patchCord4(waveformMod1, 0, waveformMod2, 1);
    // GUItool: end automatically generated code
    
    
    
    
    // the setup routine runs once when you press reset:
    void setup() {
    
      AudioMemory(256);
    
      int masterVolume = 1;
      waveformMod1.begin(WAVEFORM_SQUARE);
      waveformMod2.begin(WAVEFORM_PULSE);
      waveformMod1.offset(1);
      waveformMod1.amplitude(masterVolume);
      waveformMod2.amplitude(masterVolume);
    }
    
    // the loop routine runs over and over again forever:
    void loop() {
      float knob_1 = 0.2;
      float knob_2 = 0.8;
      float pitch1 = pow(knob_1, 2);
      // float pitch2 = pow(knob_2, 2);
      waveformMod1.frequency(10 + (pitch1 * 50));
      waveformMod2.frequency(10 + (knob_2 * 200));
      waveformMod1.frequencyModulation(knob_2 * 8 + 3);
    }

    Click image for larger version. 

Name:	graph.jpg 
Views:	19 
Size:	64.5 KB 
ID:	26440

  2. #2
    Senior Member
    Join Date
    Jul 2020
    Posts
    1,456
    Its extremely simple, the audio objects are run in the order they are declared, once each. No dependency analysis is currently done so that if
    you declare a destination before its source, that object gets the block from the last run (typically that's a 2.9ms latency hit). If you
    have a linear dataflow and declare the objects in that same order, no unnecessary latency happens. You always have to have at least one
    block of latency for any cycle of course.

    Currently the objects are chained linearly on their next_update link (see AudioStream.h in cores/teensy4/ for instance). There's a
    comment about replacing this with proper dataflow analysis. The Audio lib tool however knows to generate sensible ordering for
    objects and connections AFAICT so its only an issue if you hand-create objects in strange orders.

  3. #3
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,368
    Quote Originally Posted by MarkT View Post
    Its extremely simple, the audio objects are run in the order they are declared, once each.
    AFAIK, it is the sequence of connections that defines the sequence of executions, not the sequence of object declarations.

    Edit: best to check always SW
    It seems MarkT is correct and the sequence of declarations is driving the sequence of execution
    In AudioStream class (AudioStream.h) there is this snippet
    Code:
    			// add to a simple list, for update_all
    			// TODO: replace with a proper data flow analysis in update_all
    			if (first_update == NULL) {
    				first_update = this;
    			} else {
    				AudioStream *p;
    				for (p=first_update; p->next_update; p = p->next_update) ;
    				p->next_update = this;
    			}
    Consequently MarkT is correct
    Last edited by WMXZ; 11-09-2021 at 08:27 AM.

  4. #4
    Junior Member
    Join Date
    Oct 2021
    Location
    Scotland
    Posts
    7
    Thanks both, very much appreciated. So to confirm, in the following example,

    Code:
    AudioConnection          patchCord1(waveformMod2, 0, waveformMod1, 0);
    AudioConnection          patchCord2(waveformMod2, 0, waveformMod3, 0);
    patchCord1 is declared first, and so the processing order is: waveformMod2 then waveformMod1, then waveformMod3 (as waveformMod2 has already been processed) - waveformMod1 and waveformMod3 will both use the block from waveformMod2 that was just processed?

  5. #5
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,368
    see edit in post#3

  6. #6
    Junior Member
    Join Date
    Oct 2021
    Location
    Scotland
    Posts
    7
    Ah got it, thanks! This happens in the constructor - first_update is a static variable so will get filled in when the first AudioStream is initialised (and won't be null subsequently).

    Still trying to build a mental model of it all, interesting stuff!

  7. #7
    Senior Member
    Join Date
    Oct 2015
    Location
    Vermont, USA
    Posts
    315
    Quote Originally Posted by hemmer View Post
    Thanks both, very much appreciated. So to confirm, in the following example,

    Code:
    AudioConnection          patchCord1(waveformMod2, 0, waveformMod1, 0);
    AudioConnection          patchCord2(waveformMod2, 0, waveformMod3, 0);
    patchCord1 is declared first, and so the processing order is: waveformMod2 then waveformMod1, then waveformMod3 (as waveformMod2 has already been processed) - waveformMod1 and waveformMod3 will both use the block from waveformMod2 that was just processed?
    For other people coming to this thread later, I just wanted to be clear...no, the code above is not what sets the order that the calculations are performed. What sets the order ( per post #2 and the revised #3) is the order that the audio classes are instantiated (created), not the AudioConnections.

    The instantiation of the audio classes is not shown in the snippet above, but they're presumably created as: waveformMod1, waveformMod2, and then waveformMod3. This will also be the order in which they will be updated. Because waveformMod1 is getting called before the data from waveformMod2 is ready, waveformMod1 will end up using audio data from the previous cycle. Hence, this introduces an extra latency in the overall processing. Usually, this isn't a big deal for most uses.

    Chip

  8. #8
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    248
    And for further future reference, if Paul pulls in my Dynamic Audio Objects (as he’s said he might), the execution order then will be governed by the “physical” connection order. Though it may turn out the library gets it sub-optimal for complex cases…

    Cheers

    Jonathan

  9. #9
    Junior Member
    Join Date
    Oct 2021
    Location
    Scotland
    Posts
    7
    Yeah good to clarify. Quote was from before the other poster's edit (I can't edit that post any more). Thanks all!

  10. #10
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    9,613
    Quote Originally Posted by h4yn0nnym0u5e View Post
    the execution order then will be governed by the “physical” connection order.
    That means some Sketches will not work anymore, as they do now? (re: timing) That's incompatible.
    And will that still work with Teensy LC without using more Flash and RAM? The LC is very limited..

  11. #11
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    248
    I wouldn't rule out some sketches behaving differently, but I would guess that a dependency on exact execution order is rare. If a problem is reported then it should be possible to restore the original behaviour for static designs. For dynamic designs, assuming we take instantiation order as the equivalent of definition order, it would probably be counter-productive, because you'd almost certainly have objects instantiated after the output objects, which would introduce extra 2.9ms latency and audio block use.

    Clearly more Flash and RAM will be needed. Whether that impacts LC designs materially remains to be seen: at the moment I haven't even touched the Teensy 3 family's AudioStream code, so it's academic. It's probably also a blocker to early deployment of Dynamic Audio Objects

    I did say further future...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •