Roadmap "Dynamic Updates": any effort going on?

h4yn0nnym0u5e

Well-known member
Hi folks

Just got my first Teensy (4.1, start at the top, right?) and dead excited about the audio possibilities. I've toyed with the Nord Modular kit in the past, and this looks very like it in concept, but so far without the (semi-) live editing and patch saving capabilities. One fundamental limitation seems to be the fact that all AudioStream and AudioConnection instances are currently constrained to be statically allocated, though at least it appears the connections can be re-patched on the fly.

Before I jump in with both feet and try to get these a bit more dynamic, (a) have I failed to spot a reason why it can never be done, and (b) is anyone else already working on this? I've scanned this forum and couldn't immediately spot anything, but I guess it'd be easy to miss with the number of posts there are here. I'm hoping (a) isn't the case - the section at https://www.pjrc.com/teensy/td_libs_AudioRoadmap.html relating to this suggests that there's some chance...

Cheers

Jonathan
 
There is this dynamic matrix switch,
unfortunately I haven't tested it,
but see no reason why it should not work.

https://forum.pjrc.com/threads/59212-Audio-Router-Module?p=271498&viewfull=1#post271498

It is represented this way (4 input, 4 output cross-switch):
CrossPointSwitch.png

note. when changing the path pop and clicks can appear
that's because it don't contain any fade out/in functionality
so when changing the path make sure that the output is muted.

there could be a similar switch using fade out/in
but I have no time to do that right now.

but the fade effect could be used on each output to accomplish that.

the AudioCrosspointSwitch<inputs,outputs> is for example initialized with:

AudioCrosspointSwitch<4,4> crossSwitch;

that will create a 4 input 4 output cross switch like in the image above.


Also if you need to mix different inputs to one output you will need this (made by macaba):
https://forum.pjrc.com/threads/43752-Crosspoint-Mixer
code at
https://forum.pjrc.com/threads/4137...-on-Teensy-3-6?p=142049&viewfull=1#post142049

that is hardcoded to 16 input 16 output

also I have done a prototype n-input n-output Crosspoint Mixer
https://forum.pjrc.com/threads/43752-Crosspoint-Mixer?p=275889&viewfull=1#post275889
 
Last edited:
Thanks manicksan. This would let one switch the connections on the fly (at the cost of considerable complexity!), but as far as I can tell the AudioConnections can be changed anyway, though of course at the cost of potential glitches. They can also be created and destroyed, as they appear to have a proper constructor and destructor. The same isn't true of the AudioStream class, as it doesn't have a destructor, so you can't go from:
patch1.jpg
to
patch2.jpg
without re-compiling the code, though I think you can go to:
patch3.jpg
with the "orphaned" blocks consuming little to no CPU, and without having to re-compile.

This would be a first step on the road to being able to make edits in the Audio System Design Tool and auditioning them live in real time, saving patches to SD card and recall them on the fly, and other neat things.

Cheers

Jonathan
 
I'm interested in this topic as well. I'm working on a synth now and it would be great if I could make the sound engine "swappable" like an OP-1.

Manicksen, is your dymanic matrix switch different then using 4 mixers and setting the gains equal to 0 and 1 as desired?
 
Thanks for that thread link, houtson. I knew it could be done but it drew my attention to a couple of interesting points... Sounds from TigerBalm2’s post as if there would be interest in this ... and I’m not getting a lot of posts saying “I’m working on it” or “can’t be done”, so I may have a clear field!

I’m getting familiar with the Audio ecosystem by creating an old-school exponential envelope module: when it’s done then I’ll dive into this

Cheers

Jonathan
 
@h4yn0nnym0u5e
Creating and using dynamically/runtime created
Audio Objects would need some kind of object manager and/or some modification's to the whole Audio library.
It would be possible and I have some ideas how it can be done but that would require a whole separate thread.

@TigerBalm2
The dynamic-sized matrix switch is much easier to manage and use than a bunch of Mixer objects.

@houtson
Dynamically changing the connections have some drawbacks + bugs so that is not yet a option.
 
@h4yn0nnym0u5e
Creating and using dynamically/runtime created
Audio Objects would need some kind of object manager and/or some modification's to the whole Audio library.
It would be possible and I have some ideas how it can be done but that would require a whole separate thread.

@TigerBalm2
The dynamic-sized matrix switch is much easier to manage and use than a bunch of Mixer objects.

@houtson
Dynamically changing the connections have some drawbacks + bugs so that is not yet a option.
Sure, it's understood that the Audio core needs changes in order to construct and destroy Audio objects and their connections at runtime - that's absolutely the point of this thread, to check no-one else has tried :D The key factor seems to be that the AudioStream class doesn't have a destructor, and has links which will break the system if you simply delete an instantiation, and that's what I'd work on first because no-one can do anything without that. So I think this could be the thread for Audio core modification discussion.

Creating an object manager could then be done at the application level, though it might also be possible and preferred to define an AudioPatch class, say, which wraps things in a friendlier way. Haven't thought that far. A nice interface to MIDI or other source of control is also needed, again I haven't explored that. A new thread for that might be helpful, indeed

Can you expand on the "drawbacks + bugs" you know of with dynamic use of the AudioConnection class? I looked through the old thread Paul linked to, which did suggest varying degrees of success. One things that's probably an issue is that any disconnection is likely to cause an audio glitch, but I have a few ideas about that (an ephemeral "de-glitch" class, re-use of a (growing?) pool of AudioConnection objects with de-glitch on disconnection, muting audio if a connection is destructed without disconnecting first, etc. etc.).

Cheers

Jonathan
 
There are these threads:

https://forum.pjrc.com/threads/62166-Dynamic-Audio-Connection-Bug

https://forum.pjrc.com/threads/64446-Dynamic-Audio-Connection-Bug-A-new-one!

In the "a new one" specially check post #5

https://www.pjrc.com/teensy/td_libs_AudioConnection.html
In there it's mentioned that if you try to connect many connections to one input only the last one is used, that would mean that there is no need for doing any disconnect before doing a connect to any input, only if a input should be "unused".
 
If you want to make complicated and/or huge designs
maybe this can help:
https://forum.pjrc.com/threads/65740-Audio-System-Design-Tool-update?highlight=Audio+design+tool

That also makes it possible to use custom audio objects like the dynamic switch matrix above, by using a special node called AudioStreamObject that can be defined to have any amount of inputs/outputs.

There is no current support for c++ template type instances but it's on my roadmap.

The only dynamic thing that is implemented is my version of AudioMixer as that is required by the audio object array functionality.
 
Brilliant, thanks so much manicksan! There's a lot to digest there, but the first quick look I just gave it suggests it's all going to be really useful.

[I "need" to run a few more tests on the exponential envelope I'm working on, then I'll take a closer look]
 
I haven't run into any difficulties dynamically creating and patching audio objects, but I just do it once at startup - I don't change the connections arbitrarily during the lifetime of the program. (Is there something in the library that would make that a problem?)

My use-case is I have 32 voices, so I didn't want to have a huge and unmanageable audio design tool graph, and also as I built up and adjusted my voice design I didn't want to have to make the same change in 32 places! :)

So I made one voice class that makes it's own objects (oscillators, filter and envelopes) and connections, then I instantiate that 32 times. Then I instantiate a few mixers to join everything up and connect it to the LFO and effects and amp etc.

Works like a charm.
 
Thanks MarkT. I’ve seen (and even posted on!) that thread, though not 100% sure a fix is pending as such, though a problem has definitely been identified. I’m actually in the throes of playing with a fork of the Audio core right now, so may do a pull request when I’m done. I don’t much like the connect() code, I may tweak that too.

Cheers

Jonathan
 
I haven't run into any difficulties dynamically creating and patching audio objects, but I just do it once at startup - I don't change the connections arbitrarily during the lifetime of the program. (Is there something in the library that would make that a problem?)

My use-case is I have 32 voices, so I didn't want to have a huge and unmanageable audio design tool graph, and also as I built up and adjusted my voice design I didn't want to have to make the same change in 32 places! :)

So I made one voice class that makes it's own objects (oscillators, filter and envelopes) and connections, then I instantiate that 32 times. Then I instantiate a few mixers to join everything up and connect it to the LFO and effects and amp etc.

Works like a charm.
Yes, as noted in the thread linked by MarkT, disconnecting doesn’t work properly. Obviously if you don’t do it, you won’t have run into any problems! Connection objects as they stand are a bit pants, as they can’t be made to do any route other than the one defined at the time they were constructed: I aim to fix that today...

Sounds like you’re doing something close to my final aim, except I want to be able to destroy some instances and construct more, probably using a different graph (otherwise there’d be no point to the destruction...).

Cheers

Jonathan
 
Very Preliminary Alpha Test Dynamic Audio Objects For Teensy 4

Hi folks

I could definitely tinker for a lot longer, but why not let a few brave souls have a crack at breaking the code in its current state - I fear it will be all too easy... You will need both of:For the cores I've modified ONLY the AudioStream.cpp and .h on teensy4; the Audio library needs copying in to mask the original, in the usual manner. The new library will not work with old core, and vice versa.

I've put a document in the library docs folder detailing what users will need to do to make use of this update:
  • a "first level" user who just wants to run the demos or other existing code should see no difference at all
  • a "second level" user who wants to create and destroy AudioStream and AudioConnection objects should find they can do so
  • a "third level" user who writes or edits library objects now has to provide a working destructor: I think I've done this for most existing objects, but not all are tested

There is a new demo, PlaySynthDynamic, which is based on PlaySynthMusic but uses Bach's Brandenburg Concerto #3, first movement as the music, and more importantly implements 4 moderately-simple patches giving 7 timbres, which the various parts use in turn, with voices being created and destroyed as needed. There are options for text or graphical output to show voice, processor and memory use on the fly: changing which is in use is not well-documented, but if you can't figure at least some of it out then you probably shouldn't be using the code in this state anyway!

Known shortcomings:
  • memory supplied "externally" (e.g. for chorus effect) has to be dealt with at the application level
  • AudoEffectDelayExternal needs an update to give it a proper destructor which frees the allocated external memory: I don't understand it well enough to do that :(

Let the mayhem commence...

Cheers

Jonathan
 
The Audio Core defines the base classes on which the Audio Library is built, and is stored elsewhere. So on my Windows machine the core is in C:\Program Files (x86)\Arduino\hardware\teensy\avr\cores\teensy4\AudioStream.cpp and .h, and the Audio library is in C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Audio.cpp and .h. You can "mask" the library by having your own local one installed, but not the core, you just have to replace that (keeping a safety copy of the old one, of course!). As noted above, I've only modified the core for Teensy 4.x so far, as I don't have a Teensy 3 to test on. Shouldn't be hard to do, but at the moment I don't have much incentive.

Most of the changes required for Dynamic Audio Objects are to the core itself, but the library needed minor tweaks to many files to ensure that any audio blocks allocated to an AudioStream object are freed up if the object is destroyed.

Cheers

Jonathan
 
Ok, before digging too deep into this new code, let me ask a question first. My application is basically a sampler that plays back some background music and mixes in samples dynamically on request.
I hope I can get all samples into the onboard ROM but I might need an SPI flash for it. That should be possible - right?

Thanks,
Dan
 
Ok, before digging too deep into this new code, let me ask a question first. My application is basically a sampler that plays back some background music and mixes in samples dynamically on request.
I hope I can get all samples into the onboard ROM but I might need an SPI flash for it. That should be possible - right?

Thanks,
Dan

Hi Dan

Yes, that should be possible. TBH I've not used sample playback much, but I've seen quite a lot of wisdom around here on the pitfalls, especially the occasionally slow response time on SD cards (not a problem if you're using SPI flash, of course).

While I do want people to use this new library, it may be you can do what you want within the old one. You'll certainly be able to do some initial test code, then adopt the Dynamic Audio Objects if they're needed to achieve your final goal. DAO is designed to be backward-compatible with the existing static library, so any test code you write won't need changing just because you adopted it.

Cheers

Jonathan
 
Thanks for the feedback. Yes, it might also be possible with the existing library. The problem is mostly the fact the the mixers and muxes work only for up to 4 sources which makes switching between several hundreds of samples quite complicated. But I might have a look if there is a major performance impact when using 100 mixers.
 
Thanks for the feedback. Yes, it might also be possible with the existing library. The problem is mostly the fact the the mixers and muxes work only for up to 4 sources which makes switching between several hundreds of samples quite complicated. But I might have a look if there is a major performance impact when using 100 mixers.

I think you should only need as many mixer inputs as you have simultaneously playing samples - all the sample player objects let you nominate the sample to play, either by file name or memory address, so for a cost of five 4-input mixers you can play up to 16 samples selected from an arbitrarily large set. Mixers are cheap, especially if the gain on an unused input is set to zero. On the other hand, 100 mixers is a lot! (It's also not a well-optimised number: cascading 85 4-input mixers in four layers will give you 256 inputs). You'll have to be super-careful to set the gains correctly to avoid clipping while keeping your samples properly balanced. Plus, 256 16-bit channels at 44.1kHz results in a data rate of just over 180Mbits/s, which I believe may be ... tricky. But I don't have direct experience, as noted above.

If you did end up with a need for a super-mixer, you might want to check out @manicksan's project for a template-based mixer - not sure how best to get that, but I expect he'll be along in a minute...
 
Ok, then I misunderstood this. If I can easily switch samples while another player is still running, that should be enough already. I think I have to start experimenting with this.

(BTW: The 100 mixers where just a rough guess ;-)
 
The simplest way right now is to download this single source file:
https://raw.githubusercontent.com/manicken/teensy4polysynth2/main/src/theMixer.h

and save it to the same folder of your sketch/project.

You can also open the link and copy all text and then paste it into a new file/tab in your project/sketch.


then use #include "theMixer.h" in the header of your file where you want to use it.

it's then instanced with:
AudioMixer<80> mainMixer;

which in this case will generate a 80 input mixer.

This mixer also have a gain function that only takes one parameter,
which sets all gains to the same value.



I hope that one day Paul will include the modified mixer
available at:
https://github.com/manicken/Audio/tree/templateMixer

this is backwards compatible with the old AudioMixer4 by using:
#define AudioMixer<4> AudioMixer4
 
Back
Top