Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 21 of 21

Thread: Inner workings of the audio library

  1. #1
    Junior Member
    Join Date
    Sep 2018
    Posts
    9

    Inner workings of the audio library

    Hello folks!

    I'm wondering if there is a writeup about the inner workings of the audio library. I'm very interested in realtime audio programming and would therefore like to know how exactly it works.

    Are there any resources or do I need to dig deep into the Audio librarys source code?

    Thank you

  2. #2
    Senior Member+ Theremingenieur's Avatar
    Join Date
    Feb 2014
    Location
    Colmar, France
    Posts
    2,590
    To understand every little detail, you would have to gig into the source code. But before, you should experiment with the audio design tool on the PJRC website. The explanations and details about every audio object in the right sidebar might give you a first rough understanding. Then, there is the documentation about creating new objects for the audio library...

    And still about real-time: The audio library does its processing in blocks of 128 samples which means that there is an inherent delay of at least 2.9ms.

  3. #3
    Junior Member
    Join Date
    Sep 2018
    Posts
    9
    Is the library designed in a dsp manner? I guess so because thats what the objects represent right? Modules you can connect.

    I think"real time dsp" is never real time right? You are always feeding data into a circular buffer that is then read from the output. If I would like to understand the audio guts would it need me to just look at the Audio library or do I need anything else? I guess the audio library is just triggered in a while loop. Do you know at which frequency?

  4. #4
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    7,089
    "realtime" depends on the definition of "realtime".. with 44khz samplerate: What is realtime? If it's a defined time to react to external event... sure.. if it means "fast"...sure..but what is fast?
    Excuse my philosophical excursion

    The frequency is the time to play 128(default value) samples at 44.1Khz(default).
    Take a look at the sources. Begin with a very simple object.

  5. #5
    Senior Member+ Theremingenieur's Avatar
    Join Date
    Feb 2014
    Location
    Colmar, France
    Posts
    2,590
    Quote Originally Posted by adge View Post
    Is the library designed in a dsp manner? I guess so because thats what the objects represent right? Modules you can connect.

    I think"real time dsp" is never real time right? You are always feeding data into a circular buffer that is then read from the output. If I would like to understand the audio guts would it need me to just look at the Audio library or do I need anything else? I guess the audio library is just triggered in a while loop. Do you know at which frequency?
    There is no while loop. It’s all about objects and update responsibilities. The output objects consume data at a fixed timing, the sample rate. As one buffer is consumed, an update request is triggered at the preceding object in the chain to fill the buffer up again. Thus, everything is rather event driven on a request and following update strategy.

  6. #6
    Senior Member+ Theremingenieur's Avatar
    Join Date
    Feb 2014
    Location
    Colmar, France
    Posts
    2,590
    Quote Originally Posted by Frank B View Post
    "realtime" depends on the definition of "realtime".. with 44khz samplerate: What is realtime? If it's a defined time to react to external event... sure.. if it means "fast"...sure..but what is fast?
    Excuse my philosophical excursion

    Internal: There is a global instance which calls "update() for every object. The in- and outputs have additional timers (ISRs) and dma
    While you take a philosophical approach, I’ll take a discrete academical approach: Real time means for me that any input change would have an immediate effect on the very next output sample.
    Last edited by Frank B; 09-22-2018 at 02:49 PM. Reason: oops

  7. #7
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    7,089
    In this case: no, not realtime. It has immedate effect to the next played 128-sample block, but not the next sample.
    It's possible to reduce the blocksize to 16 samples (increases overhead slightly).

  8. #8
    Senior Member+ Theremingenieur's Avatar
    Join Date
    Feb 2014
    Location
    Colmar, France
    Posts
    2,590
    I’m amazed that you are back, Frank. I really missed these little exchanges!

  9. #9
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    7,089

  10. #10
    Junior Member
    Join Date
    Sep 2018
    Posts
    9
    Quote Originally Posted by Theremingenieur View Post
    There is no while loop. It’s all about objects and update responsibilities. The output objects consume data at a fixed timing, the sample rate. As one buffer is consumed, an update request is triggered at the preceding object in the chain to fill the buffer up again. Thus, everything is rather event driven on a request and following update strategy.
    What? The processor is executing commands. One after another right? There has to be some "update audio" hook that is called every cycle...or not?
    This is an unconventional approach right? I've never seen....well I have not seen much but I always thought that there is something repeatedly going on. Hierarchical very deep. And from there e.g. a sine is processed and written into a circular buffer. And something else just reads from this buffer. Any resources so I could teach myself on that subject? Low level wise?

    And which .ccp file in the audio library starts everything?

  11. #11
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    7,089
    Arduino\hardware\teensy\avr\cores\teensy3\AudioStr eam.cpp in the core - not library.

    But you don't need to know that in this detail - for the beginning, it's sufficiant to look at a simple audio object. play_memory for example.

  12. #12
    Junior Member
    Join Date
    Sep 2018
    Posts
    9
    Quote Originally Posted by Frank B View Post
    Arduino\hardware\teensy\avr\cores\teensy3\AudioStr eam.cpp in the core - not library.

    But you don't need to know that in this detail - for the beginning, it's sufficiant to look at a simple audio object. play_memory for example.
    Yeah I know. I already messed around with the gui audio design too. Its super easy to glue some parts together. However I always have the urge to understand how the glue and the parts themselves work

  13. #13
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    7,089
    Well, then audiostream.cpp will help you

  14. #14
    Junior Member
    Join Date
    Sep 2018
    Posts
    9
    Does anybody know how I would do wave sequencing synthesis with the audio library? Sequencing parts of waveforms in a row that is then played as a whole... The old nintendo worked like this

  15. #15
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    7,089
    Do you mean playing parts of predefined wavesforms in memory?
    I think the play-memory can be modified to do this.

  16. #16
    Junior Member
    Join Date
    Sep 2018
    Posts
    9
    Quote Originally Posted by Frank B View Post
    Do you mean playing parts of predefined wavesforms in memory?
    I think the play-memory can be modified to do this.
    This is what I'm talking about:
    https://www.youtube.com/watch?v=jvIz...&frags=pl%2Cwn

    But yeah, seqencing single or multiple cycles of different waveforms together and playing them.

  17. #17
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,061
    You can easily get an effect similar to this diagram from 3:18 in the video.

    Click image for larger version. 

Name:	sc.jpg 
Views:	42 
Size:	31.9 KB 
ID:	14789

    To accomplish the cross fading, you'd use a design similar to this:

    Click image for larger version. 

Name:	sc.png 
Views:	49 
Size:	29.9 KB 
ID:	14790

    In your code, you'd configure those sample players or waveform objects to create the short sounds you want.

    Set all the mixer channels to 1.0 gain, since you'll be using the faders to attenuate the signals so the add up to at most 1.0 before mixing.

    Then when you want to cross fade between them with a 40 millisecond transition time, you'd use code like this:

    Code:
      AudioNoInterrupts();
      fade1.fadeOut(40);
      fade3.fadeIn(40);
      AudioInterrupts();
    The important step here is AudioNoInterrupts() surrounding the setting of both fade objects, so they will be certain to begin their transitions at the same instant.

    If you want the waveform to start at a precise point within its cycle, you'd also put that include the code protected by AudioNoInterrupts(), so the fading in waveform synthesis begins at exactly the designed phase angle (0 to 359.999 degrees) as the fading in begins.

    Code:
      AudioNoInterrupts();
      fade1.fadeOut(40);
      waveform1.phase(180.0);
      fade3.fadeIn(40);
      AudioInterrupts();
    Of course, you can also configure for any of the 9 waveforms and their various settings. If using a modulated waveform, you can configure it and the waveform that's modulating its frequency, phase or shape (if pulse or variable triangle) so you have exact control over exactly how the waveform begins at the exact moment the 40 ms (or whatever time you configure) cross fade begins.

  18. #18
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,061
    Having said all that, I believe it's important to emphasize how tedious and painstaking this sort of sound design can be. The video mentions 5 weeks. If you do manage to create something this way that sounds good, I really hope you'll share.

  19. #19
    Junior Member
    Join Date
    Sep 2018
    Posts
    9
    Thank you Paul this information is extremely helpful. We'll see if I can come up with something interesting. My plan was to build a small synth that is capable of a few different sound synthesizing methods. Mostly just for learning.

    I still have one Question though. Can I somehow specify how many cycles a wave should be played before faded out? Or should I just calculate the play time with the waves frequency it needs to pass e.g. 3 cycles?

    Anyway thank you for your information.

  20. #20
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,061
    Quote Originally Posted by adge View Post
    Can I somehow specify how many cycles a wave should be played before faded out?
    No, not directly.

    Using this method, you would wait (by delay or elapsedMillis or other ways of measuring elapsed time) and then call the fadeOut(ms) function at the right moment.

    If you want it all to happen automatically, you could configure the envelope object to have a zero (silent) sustain level, and use the attach, hold and decay times to implement your intended fade in and out.

  21. #21
    Junior Member
    Join Date
    Sep 2018
    Posts
    9
    Allright. I find this stuff super interesting. Are there things that are not, or hardly achievable with this library? Not really right since we can create custom objects too.
    Are all audio libraries designed like that? It must be extremely tedious to come up with something complex like that that brings you maximum functionality and freedom without having to redesign everything. So it can really fit to most needs.

    And how does AudioNoInterrupts() work? I guess there is no multithreading right? So how can it perform two actions simultaneously?
    The documentation says it disables the audio libraries update interrupt. Whatever that means. Why would you intereupt the update step of the audio library?

    Isn‘t there something like:

    while (teensy)
    {
    handle events
    process libraries (e.g. audio)
    }
    Last edited by adge; 09-25-2018 at 02:45 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •