Forum Rule: Always post complete source code & details to reproduce any issue!
Page 1 of 2 1 2 LastLast
Results 1 to 25 of 31

Thread: Roadmap "Dynamic Updates": any effort going on?

  1. #1
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124

    Roadmap "Dynamic Updates": any effort going on?

    Hi folks

    Just got my first Teensy (4.1, start at the top, right?) and dead excited about the audio possibilities. I've toyed with the Nord Modular kit in the past, and this looks very like it in concept, but so far without the (semi-) live editing and patch saving capabilities. One fundamental limitation seems to be the fact that all AudioStream and AudioConnection instances are currently constrained to be statically allocated, though at least it appears the connections can be re-patched on the fly.

    Before I jump in with both feet and try to get these a bit more dynamic, (a) have I failed to spot a reason why it can never be done, and (b) is anyone else already working on this? I've scanned this forum and couldn't immediately spot anything, but I guess it'd be easy to miss with the number of posts there are here. I'm hoping (a) isn't the case - the section at https://www.pjrc.com/teensy/td_libs_AudioRoadmap.html relating to this suggests that there's some chance...

    Cheers

    Jonathan

  2. #2
    Senior Member manicksan's Avatar
    Join Date
    Jun 2020
    Location
    Sweden
    Posts
    294
    There is this dynamic matrix switch,
    unfortunately I haven't tested it,
    but see no reason why it should not work.

    https://forum.pjrc.com/threads/59212...l=1#post271498

    It is represented this way (4 input, 4 output cross-switch):
    Name:  CrossPointSwitch.png
Views: 602
Size:  16.4 KB

    note. when changing the path pop and clicks can appear
    that's because it don't contain any fade out/in functionality
    so when changing the path make sure that the output is muted.

    there could be a similar switch using fade out/in
    but I have no time to do that right now.

    but the fade effect could be used on each output to accomplish that.

    the AudioCrosspointSwitch<inputs,outputs> is for example initialized with:

    AudioCrosspointSwitch<4,4> crossSwitch;

    that will create a 4 input 4 output cross switch like in the image above.


    Also if you need to mix different inputs to one output you will need this (made by macaba):
    https://forum.pjrc.com/threads/43752-Crosspoint-Mixer
    code at
    https://forum.pjrc.com/threads/41371...l=1#post142049

    that is hardcoded to 16 input 16 output

    also I have done a prototype n-input n-output Crosspoint Mixer
    https://forum.pjrc.com/threads/43752...l=1#post275889
    Last edited by manicksan; 04-07-2021 at 11:39 AM. Reason: image missing

  3. #3
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Thanks manicksan. This would let one switch the connections on the fly (at the cost of considerable complexity!), but as far as I can tell the AudioConnections can be changed anyway, though of course at the cost of potential glitches. They can also be created and destroyed, as they appear to have a proper constructor and destructor. The same isn't true of the AudioStream class, as it doesn't have a destructor, so you can't go from:
    Click image for larger version. 

Name:	patch1.jpg 
Views:	55 
Size:	29.2 KB 
ID:	24360
    to
    Click image for larger version. 

Name:	patch2.jpg 
Views:	53 
Size:	16.4 KB 
ID:	24361
    without re-compiling the code, though I think you can go to:
    Click image for larger version. 

Name:	patch3.jpg 
Views:	31 
Size:	27.8 KB 
ID:	24362
    with the "orphaned" blocks consuming little to no CPU, and without having to re-compile.

    This would be a first step on the road to being able to make edits in the Audio System Design Tool and auditioning them live in real time, saving patches to SD card and recall them on the fly, and other neat things.

    Cheers

    Jonathan

  4. #4
    Junior Member
    Join Date
    Mar 2021
    Posts
    18
    I'm interested in this topic as well. I'm working on a synth now and it would be great if I could make the sound engine "swappable" like an OP-1.

    Manicksen, is your dymanic matrix switch different then using 4 mixers and setting the gains equal to 0 and 1 as desired?

  5. #5
    Senior Member houtson's Avatar
    Join Date
    Aug 2015
    Location
    Scotland
    Posts
    201
    hi h4yn0nnym0u5e, This old thread talks about dynamic connections, might be useful ? cheers, Paul

  6. #6
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Thanks for that thread link, houtson. I knew it could be done but it drew my attention to a couple of interesting points... Sounds from TigerBalm2ís post as if there would be interest in this ... and Iím not getting a lot of posts saying ďIím working on itĒ or ďcanít be doneĒ, so I may have a clear field!

    Iím getting familiar with the Audio ecosystem by creating an old-school exponential envelope module: when itís done then Iíll dive into this

    Cheers

    Jonathan

  7. #7
    Senior Member manicksan's Avatar
    Join Date
    Jun 2020
    Location
    Sweden
    Posts
    294
    @h4yn0nnym0u5e
    Creating and using dynamically/runtime created
    Audio Objects would need some kind of object manager and/or some modification's to the whole Audio library.
    It would be possible and I have some ideas how it can be done but that would require a whole separate thread.

    @TigerBalm2
    The dynamic-sized matrix switch is much easier to manage and use than a bunch of Mixer objects.

    @houtson
    Dynamically changing the connections have some drawbacks + bugs so that is not yet a option.

  8. #8
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Quote Originally Posted by manicksan View Post
    @h4yn0nnym0u5e
    Creating and using dynamically/runtime created
    Audio Objects would need some kind of object manager and/or some modification's to the whole Audio library.
    It would be possible and I have some ideas how it can be done but that would require a whole separate thread.

    @TigerBalm2
    The dynamic-sized matrix switch is much easier to manage and use than a bunch of Mixer objects.

    @houtson
    Dynamically changing the connections have some drawbacks + bugs so that is not yet a option.
    Sure, it's understood that the Audio core needs changes in order to construct and destroy Audio objects and their connections at runtime - that's absolutely the point of this thread, to check no-one else has tried The key factor seems to be that the AudioStream class doesn't have a destructor, and has links which will break the system if you simply delete an instantiation, and that's what I'd work on first because no-one can do anything without that. So I think this could be the thread for Audio core modification discussion.

    Creating an object manager could then be done at the application level, though it might also be possible and preferred to define an AudioPatch class, say, which wraps things in a friendlier way. Haven't thought that far. A nice interface to MIDI or other source of control is also needed, again I haven't explored that. A new thread for that might be helpful, indeed

    Can you expand on the "drawbacks + bugs" you know of with dynamic use of the AudioConnection class? I looked through the old thread Paul linked to, which did suggest varying degrees of success. One things that's probably an issue is that any disconnection is likely to cause an audio glitch, but I have a few ideas about that (an ephemeral "de-glitch" class, re-use of a (growing?) pool of AudioConnection objects with de-glitch on disconnection, muting audio if a connection is destructed without disconnecting first, etc. etc.).

    Cheers

    Jonathan

  9. #9
    Senior Member manicksan's Avatar
    Join Date
    Jun 2020
    Location
    Sweden
    Posts
    294
    There are these threads:

    https://forum.pjrc.com/threads/62166...Connection-Bug

    https://forum.pjrc.com/threads/64446...-Bug-A-new-one!

    In the "a new one" specially check post #5

    https://www.pjrc.com/teensy/td_libs_...onnection.html
    In there it's mentioned that if you try to connect many connections to one input only the last one is used, that would mean that there is no need for doing any disconnect before doing a connect to any input, only if a input should be "unused".

  10. #10
    Senior Member manicksan's Avatar
    Join Date
    Jun 2020
    Location
    Sweden
    Posts
    294
    If you want to make complicated and/or huge designs
    maybe this can help:
    https://forum.pjrc.com/threads/65740...io+design+tool

    That also makes it possible to use custom audio objects like the dynamic switch matrix above, by using a special node called AudioStreamObject that can be defined to have any amount of inputs/outputs.

    There is no current support for c++ template type instances but it's on my roadmap.

    The only dynamic thing that is implemented is my version of AudioMixer as that is required by the audio object array functionality.

  11. #11
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Brilliant, thanks so much manicksan! There's a lot to digest there, but the first quick look I just gave it suggests it's all going to be really useful.

    [I "need" to run a few more tests on the exponential envelope I'm working on, then I'll take a closer look]

  12. #12
    I haven't run into any difficulties dynamically creating and patching audio objects, but I just do it once at startup - I don't change the connections arbitrarily during the lifetime of the program. (Is there something in the library that would make that a problem?)

    My use-case is I have 32 voices, so I didn't want to have a huge and unmanageable audio design tool graph, and also as I built up and adjusted my voice design I didn't want to have to make the same change in 32 places! :-)

    So I made one voice class that makes it's own objects (oscillators, filter and envelopes) and connections, then I instantiate that 32 times. Then I instantiate a few mixers to join everything up and connect it to the LFO and effects and amp etc.

    Works like a charm.

  13. #13
    Senior Member
    Join Date
    Jul 2020
    Posts
    1,253
    There's a fix pending to the dynamic update stuff I believe, I suggest follow this thread for updates, or to
    try the patch directly:
    https://forum.pjrc.com/threads/64446...Bug-A-new-one!

  14. #14
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Thanks MarkT. I’ve seen (and even posted on!) that thread, though not 100% sure a fix is pending as such, though a problem has definitely been identified. I’m actually in the throes of playing with a fork of the Audio core right now, so may do a pull request when I’m done. I don’t much like the connect() code, I may tweak that too.

    Cheers

    Jonathan

  15. #15
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Quote Originally Posted by kallikak View Post
    I haven't run into any difficulties dynamically creating and patching audio objects, but I just do it once at startup - I don't change the connections arbitrarily during the lifetime of the program. (Is there something in the library that would make that a problem?)

    My use-case is I have 32 voices, so I didn't want to have a huge and unmanageable audio design tool graph, and also as I built up and adjusted my voice design I didn't want to have to make the same change in 32 places! :-)

    So I made one voice class that makes it's own objects (oscillators, filter and envelopes) and connections, then I instantiate that 32 times. Then I instantiate a few mixers to join everything up and connect it to the LFO and effects and amp etc.

    Works like a charm.
    Yes, as noted in the thread linked by MarkT, disconnecting doesnít work properly. Obviously if you donít do it, you wonít have run into any problems! Connection objects as they stand are a bit pants, as they canít be made to do any route other than the one defined at the time they were constructed: I aim to fix that today...

    Sounds like youíre doing something close to my final aim, except I want to be able to destroy some instances and construct more, probably using a different graph (otherwise thereíd be no point to the destruction...).

    Cheers

    Jonathan

  16. #16
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Bit of a cross-posting, but dynamic connections are possibly working now, see https://github.com/h4yn0nnym0u5e/cor...n-improvements.

    In other news, manicksan, I looked at your customised Audio Design Tool, but not for anything like long enough: needs a much better look to see what you've added and how it works, but it certainly looks like a huge improvement...

    Cheers

    Jonathan

  17. #17
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124

    Very Preliminary Alpha Test Dynamic Audio Objects For Teensy 4

    Hi folks

    I could definitely tinker for a lot longer, but why not let a few brave souls have a crack at breaking the code in its current state - I fear it will be all too easy... You will need both of:
    For the cores I've modified ONLY the AudioStream.cpp and .h on teensy4; the Audio library needs copying in to mask the original, in the usual manner. The new library will not work with old core, and vice versa.

    I've put a document in the library docs folder detailing what users will need to do to make use of this update:
    • a "first level" user who just wants to run the demos or other existing code should see no difference at all
    • a "second level" user who wants to create and destroy AudioStream and AudioConnection objects should find they can do so
    • a "third level" user who writes or edits library objects now has to provide a working destructor: I think I've done this for most existing objects, but not all are tested


    There is a new demo, PlaySynthDynamic, which is based on PlaySynthMusic but uses Bach's Brandenburg Concerto #3, first movement as the music, and more importantly implements 4 moderately-simple patches giving 7 timbres, which the various parts use in turn, with voices being created and destroyed as needed. There are options for text or graphical output to show voice, processor and memory use on the fly: changing which is in use is not well-documented, but if you can't figure at least some of it out then you probably shouldn't be using the code in this state anyway!

    Known shortcomings:
    • memory supplied "externally" (e.g. for chorus effect) has to be dealt with at the application level
    • AudoEffectDelayExternal needs an update to give it a proper destructor which frees the allocated external memory: I don't understand it well enough to do that


    Let the mayhem commence...

    Cheers

    Jonathan

  18. #18
    Senior Member
    Join Date
    Feb 2017
    Posts
    423
    What exactly do you mean by the "Audio Core"? How is it different than the Audio Library?

  19. #19
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    The Audio Core defines the base classes on which the Audio Library is built, and is stored elsewhere. So on my Windows machine the core is in C:\Program Files (x86)\Arduino\hardware\teensy\avr\cores\teensy4\Au dioStream.cpp and .h, and the Audio library is in C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Audio. cpp and .h. You can "mask" the library by having your own local one installed, but not the core, you just have to replace that (keeping a safety copy of the old one, of course!). As noted above, I've only modified the core for Teensy 4.x so far, as I don't have a Teensy 3 to test on. Shouldn't be hard to do, but at the moment I don't have much incentive.

    Most of the changes required for Dynamic Audio Objects are to the core itself, but the library needed minor tweaks to many files to ensure that any audio blocks allocated to an AudioStream object are freed up if the object is destroyed.

    Cheers

    Jonathan

  20. #20
    Junior Member
    Join Date
    Jun 2021
    Posts
    3
    Ok, before digging too deep into this new code, let me ask a question first. My application is basically a sampler that plays back some background music and mixes in samples dynamically on request.
    I hope I can get all samples into the onboard ROM but I might need an SPI flash for it. That should be possible - right?

    Thanks,
    Dan

  21. #21
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Quote Originally Posted by usul27 View Post
    Ok, before digging too deep into this new code, let me ask a question first. My application is basically a sampler that plays back some background music and mixes in samples dynamically on request.
    I hope I can get all samples into the onboard ROM but I might need an SPI flash for it. That should be possible - right?

    Thanks,
    Dan
    Hi Dan

    Yes, that should be possible. TBH I've not used sample playback much, but I've seen quite a lot of wisdom around here on the pitfalls, especially the occasionally slow response time on SD cards (not a problem if you're using SPI flash, of course).

    While I do want people to use this new library, it may be you can do what you want within the old one. You'll certainly be able to do some initial test code, then adopt the Dynamic Audio Objects if they're needed to achieve your final goal. DAO is designed to be backward-compatible with the existing static library, so any test code you write won't need changing just because you adopted it.

    Cheers

    Jonathan

  22. #22
    Junior Member
    Join Date
    Jun 2021
    Posts
    3
    Thanks for the feedback. Yes, it might also be possible with the existing library. The problem is mostly the fact the the mixers and muxes work only for up to 4 sources which makes switching between several hundreds of samples quite complicated. But I might have a look if there is a major performance impact when using 100 mixers.

  23. #23
    Senior Member
    Join Date
    Apr 2021
    Location
    Cambridgeshire, UK
    Posts
    124
    Quote Originally Posted by usul27 View Post
    Thanks for the feedback. Yes, it might also be possible with the existing library. The problem is mostly the fact the the mixers and muxes work only for up to 4 sources which makes switching between several hundreds of samples quite complicated. But I might have a look if there is a major performance impact when using 100 mixers.
    I think you should only need as many mixer inputs as you have simultaneously playing samples - all the sample player objects let you nominate the sample to play, either by file name or memory address, so for a cost of five 4-input mixers you can play up to 16 samples selected from an arbitrarily large set. Mixers are cheap, especially if the gain on an unused input is set to zero. On the other hand, 100 mixers is a lot! (It's also not a well-optimised number: cascading 85 4-input mixers in four layers will give you 256 inputs). You'll have to be super-careful to set the gains correctly to avoid clipping while keeping your samples properly balanced. Plus, 256 16-bit channels at 44.1kHz results in a data rate of just over 180Mbits/s, which I believe may be ... tricky. But I don't have direct experience, as noted above.

    If you did end up with a need for a super-mixer, you might want to check out @manicksan's project for a template-based mixer - not sure how best to get that, but I expect he'll be along in a minute...

  24. #24
    Junior Member
    Join Date
    Jun 2021
    Posts
    3
    Ok, then I misunderstood this. If I can easily switch samples while another player is still running, that should be enough already. I think I have to start experimenting with this.

    (BTW: The 100 mixers where just a rough guess ;-)

  25. #25
    Senior Member manicksan's Avatar
    Join Date
    Jun 2020
    Location
    Sweden
    Posts
    294
    The simplest way right now is to download this single source file:
    https://raw.githubusercontent.com/ma...src/theMixer.h

    and save it to the same folder of your sketch/project.

    You can also open the link and copy all text and then paste it into a new file/tab in your project/sketch.


    then use #include "theMixer.h" in the header of your file where you want to use it.

    it's then instanced with:
    AudioMixer<80> mainMixer;

    which in this case will generate a 80 input mixer.

    This mixer also have a gain function that only takes one parameter,
    which sets all gains to the same value.



    I hope that one day Paul will include the modified mixer
    available at:
    https://github.com/manicken/Audio/tree/templateMixer

    this is backwards compatible with the old AudioMixer4 by using:
    #define AudioMixer<4> AudioMixer4

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •