Audio System Design Tool++ update

manicksan

Well-known member
Hi,

In July I posted here to show some changes that I had done to the "Audio System Design Tool"
https://forum.pjrc.com/threads/61630-Audio-System-Design-Tool-update-AudioMixerX-lt-n-gt

note. in the following text I will just refer "Audio System Design Tool" as "Tool"

And at that time I had just beginning to understand how the code worked,
and there was some things that I did not fully understand,
also I had made the changes on a old version of the "Tool".

After that I decided to give it a go with a newer version.

(I think this was a really good/fun way to learn about html, css and javascript)

Now the "Tool" is pretty stable with no known bugs,
except for some minor visual stuff that will be fixed soon.

at post #10
https://forum.pjrc.com/threads/64572-Simple-Way-to-add-Polyphony
I mention a ACE-editor bug, that bug is now fixed.
( that means the ACE-editor can be resized to any size)

Here is the result:
https://manicken.github.io/
source: https://github.com/manicken/manicken.github.io
MainScreenDemo.jpg

I have included my complete Poly Synth Project into the html
it's available at the rightmost menu -> Examples -> Manicken Poly Synth
that can be exported as a zip/or using the plugin (more about that down)


As you can see a lot has happened:

* Settings tab at the right sidebar
where some user customizations can be done
like project name + default include header
settings are sorted in different categories,
and subcategories.
The settings are saved with the project.

* Workspace toolbar:
common functions cut copy paste
workspace move left/right (to organize the tabs)
GUI Edit/Run (only used for the UI objects)

the toolbar can be hidden in the settings

* Drop down Import and Export Menus
ImportMenu.pngExportMenu.png

* when exporting/saving the nodes are sorted by the visible major grid columns
1 4 7
2 5 8
3 6 9

note. the UI items are never sorted

* Simple export is used for "classic" exports.
At the moment it only supports exporting of the current "tab"
and is only meant to be used when doing small and simple projects
it currently have no support for arrays and subclasses
however it exports:
IncludeDef, Variables, CodeFile, Function, AudioStreamObject and DontRemoveCodeFiles node-types.

* save/load the whole design to/from a JSON file

* create a design in parts that is then exported as different classes
(was already in my prev. version)

* export the whole class based design to a zip file
for those that don't want to use the extensions and/or
just want to make a "shippable" version
this zip also contains the JSON in human readable format.

* menu "buttons" that is not that obvious have popup descriptions.

* create code directly in the "Tool" using ACE editor
with local multilevel autocomplete,
that means it uses the current design objects to create the autocomplete list.
It also utilizes the embedded documentation to create the function lists,
with descriptions both for the objects and object functions.
Autocomplete currently don't have support for variables.

example how the autocomplete looks:
Autocomplete_MultiLevel.png

Code is edited in CodeFile and Function node types.

* Arduino IDE plugin/extension
https://github.com/manicken/arduinoAPIwebserver

* VSCODE extension
https://github.com/manicken/vscode_api_webserver

* in short the IDE extensions is used to create a direct link between the "Tool"
and the IDE:s (so that no copy/paste is needed).
The extensions send all exported classes as separate files, to make nice and tidy code files.
The extensions also make it possible to send verify/upload commands so that you can make all
development in the "Tool".
i.e. Just click export (simple/class) then click upload, and the changes are direcly applied
without even touching either Arduino IDE or VSCODE.

The extensions also capture the output of the compile output and send them to the "Tool"
visible in the "Tool" bottom output-log

Note. Arduino IDE compile verbose mode generates alot of information,
so in short: the output in the "Tool" still outputs data long after the actual compile has been finished
in the Arduino IDE.

In VSCODE the compile don't generate so much information and that is not having any "delay"
but in VSCODE the terminal capture is not working by default, and VSCODE needs to be started with
the parameter --enable-proposed-api JannikSvensson.api-webserver
(yes proposed-api is available in the standard version, even if microsoft say it's not)
the information of doing so is available at the VSCODE extension github page.

Only the Arduino IDE extension is supporting the "midi to websocket bridge"
that is used by "UI item node" midiSend commands.
This is because midi is not native in Node.js, and the node-midi (https://www.npmjs.com/package/midi)
had some version compatibility problems in VSCODE.

But I have made the Arduino IDE plugin .jar executable,
that only contains the "midi to websocket bridge".

The MIDI device selection is done in the "Tool" at the settings tab - BiDirData WebSocketBridge - MIDI

* UI nodes (directly resizeable)
is meant to be used together with the BiDirData WebSocketBridge
to control a midi device from the "Tool"
this is to make it easier to have everything in one "project".

to send data to a connected midi device the sendmidi(0x90, 60, 0x3F) command is used (note. that the parameters can be mixed types)

All UI nodes have user-selectable colors.

UI node types currently supported:

* group (is in the UI category)
can contain groups of nodes, all types are supported
can even place group into group, fully drag/drop (not like "Node-Red" group)
completely made from scratch.
the objects placed inside a group uses "global" position and not local (as expected),
this is because I did not want to screw up the positions and also make it easier to code.
the group act like a lasso-selection (but it uses a node-list to determite which nodes that is "inside")

the group can have label that is placed inside it at the top center
(in a future version it should be user placeable like in the "node-red"-group)

I plan to use the group like a exportable class/(seperate object) so that it can be used like a array.
so the checkbox "Export as class" have no function yet.

example of grouped listboxes:
GroupedListBoxes.png

* UI_Button
have seperate pressed and release send data fields (the text in the field is the raw data sent)

* UI_Slider (vertical or horizontal orientation)
(send on move or release)
have a bottom placed label that can show the current value
(the text to show is parsed using javascript eval(), that means complex formulas can be used)
have a "Send Format" field that is also parsed by the eval, example:
"midisend(0xB0,"+d.midiId+"," + d.val + ");"

H_Slider.pngV_Slider.png

* UI_Label
is a borderless node with a selectable background color

* UI_ListBox
contains a list of items and behaves like radio buttons.
when placed in a group and the group have the checkbox "Individual ListBox Mode" unchecked they all
behaves like a big listbox item (can be used when long list of items need to be splitted into many columns)
sendCommand example:
"midiSend("+(0xB0+d.midiCh)+","+d.midiId+","+d.selectedIndex+")"
ListBox.png

* UI_Piano (one octave)
this is the most advanced (by design) node yet
it's a single octave Piano-part (multiple can be stacked together to form a full size 128key keyboard)

sendCommand example:
"midiSend("+(d.keyDown+d.midiCh)+","+(d.octave*12+d.keyIndex) +",127)"

Piano.png

* UI_ScriptButton
this is the most fun node type,
it's used editable JavaScript that is executed when the button is pressed (in GUI run mode)
this can be used to automate alot of things,
I have used to automatically place the full-size keyboard above, with all sendCommands prefilled.
Also I have used it to place the sliders used to control the envelope.
And to make a beatiful rainbow equalizer slider-array.

It uses the View.js as the eval base, so that every function available in View.js can be used directly.

It can also be used for a custom export for custom structures.

* the import and export has been drop down menus with a lot of choices

* the palette have all categories closed by default (so its easier to find items)
+ automatic close of other than current selected
+ input and output have sub categories sorted by "device"(don't know what to call it)
+ used category which contains all AudioObject-types used (to make it easier to make copies of used objects)
+ 3 new categories:
1. tabs (contains class IO nodes + all class nodes)

* print function at right menu/(ctrl+p) that only uses the workspace-area as the print source

* save button (ctrl+s) to ensure that the design is saved
save notification that shows when the design is saved

* Junction Nodes (used to split many signals and to make the design easier to follow)
have two variants one that goes in normal direction L-R
and one that goes in the reverse direction

example of R-L junction usage:
ReverseJunction.png

I tried my best to make the embedded documentation as clear as possible,
but contributions is appreciated.
Or if there is something that needs more explanation, please let me know.


List of future plans (in priority order):

* tidy up the node label/text redraw code
(to make it more efficient because the getTextSize is taking alot of time)
so buffering of the textDimensions is needed for all labels/texts

* Make a very simple sequencer with the help of piano as guide and group with lines for the sequence part
the notes/actions should be done with the UI_Button (because it have both pressed(enter) and released(leave) actions)

* make group export as class
I have included two proposed designs into the html
it's available at the rightmost menu -> Examples -> GroupBasedDesign (this contains two tabs with each design)

* "exploded" export where a design can be defined in tabs but then exported as "classic"-export

* multitab code editor
 
I have now made it possible to define the main "sketch" file
as a tab in the tool
mainCodeAsTab.png

this makes it possible to split the code into different nodes
even for the main-file (it was not possible before, as everything got exported as a class)

to define this special tab
the name must be exactly main.cpp (it's case sensitive)

then when this gets exported the class definer is never used
and also there can not be any Audio nodes as they are also filtered
i.e. it gets exported with a standard Arduino sketch layout.

this new functionality makes it possible to use this tool
for ordinary programming
i.e. non Audio stuff
but in code block-style

note.
I have updated my embedded project to use the above functionality
so it's easier to see the different blocks of the main code.

Have also added an additional embedded simple example that is non Audio.
to export this kind of projects the checkbox at settings-tab
Arduino-Export-IO check At Export
needs to be unchecked.


No other updates at this time.

Edit.
Forgot to mention that to use my embedded project you need to download the instrument files located at my github
links to those files are embedded in a node called "instrument files to manually download" that is at the "Synth" tab
they are to big to include in the example (4MB) and cannot be saved to the "local storage" that the "Tool" uses.
 
Last edited:
This is a work of pure genius - rather curiously Paul Stoffregen posted on Twitter over Christmas saying he should look at doing a "multi voice" audio tool and when I tweeted back about this (your older version) he wasn't aware of it but really this should become the main audio design tool that everyone uses.

Now with an older version (around Nov/Dec) I started to move my mono synth design from the original design tool to your class based one with the N input mixer with a view to making my own synth more organised and, most importantly, polyphonic but I have to admit that I "stalled" because there were some things I couldn't quite achieve. I basically wanted to move everything to the tool editor (rather than editing the generated code after creation) but the bit it would not allow then was the main.cpp kind of "glue" where you actually instantiate the classes and the glue to connect things up (like the MIDI and USB-MIDI callbacks etc) but I see in this latest version, and especially from looking at your polysynth sample, that you have now implemented all the bits I was missing. So I can't wait to try and finish my part finished port.

(BTW as I explore your poly example with the voices having 3 osc and wavetable each it begins to look spookily like my own design!)

Oh and if anyone needs a full 128 voice GM set of wavetables by the way I have them (almost) all here: https://github.com/wrightflyer/Synth/tree/master/src/GMinstrument the core file is GMinst.cpp which binds all the samples together. I had to leave out 4 (comments identify which in the file) as they burst the 8MB in a Teensy 4.1 but as it stands it's under 4MB which easily fits in a Teensy 4.1, with a bit of trimming one could make a reduced set for Teensy 4.0 I get (I actually started with Teensy 4.0 but upgraded on Black Friday as it was easier just to by the bigger Teensy than to try and pick which instruments to discard - the likely candidates were the big piano samples but they are the very ones you'd probably want to keep!)

Oh and I have some notes in my Wiki ( https://github.com/wrightflyer/Synth/wiki/Wavetable---SoundFonts-(SF2) ) about how I found the right set of GM samples for this - ironically they are basically the Windows (well, OK Roland) ones that we've all heard when Windows MIDI was left at the default "Microsoft GS Wavetable Synth". They offer (IMHO) the best quality to sample size ratio of anything I have listened too.

PS forgot to say that with you buttons for "GUI Edit"/"GUI Run" and your "widgets" for use in that Run mode I can't help noticing that you seem to be inventing a prettier looking version of "Pure Data" ;-)

Just a thought shouldn't be just one toggling button between Edit and Run - that would then be almost identical to Ctrl-E ("Edit Mode" switch) in Pure Data or Ctrl-E ("Panel Mode") in JUCE Ctrlr.
 
Nice to hear that you like the new tool.

Thanks for sharing the 128 voice GM set,
i will implement some of them,
specially ChurchOrg1 as i play
the intro of Toccata and Fugue
also its good for
In the Hall of the Mountain King

I tried to use Pure Data but as I couldn't use the mouse scroll for the sliders,
Also though it was hard to use (and to do special things), so I left that rather quickly.

I designed my own in C# instead (as I master C# a lot)
MyCsharpMidiController.png
in the program above I have created a custom controller that I call UC_Trackbars
it makes it very easy to setup the design I have.



The reason that I have two buttons for the gui edit/run modes
is to make it easier/clearer to select the mode I wanted when
testing/debugging, also it was easier to just have two buttons
at the time.

I have also added a toggle button
And changed the other two into a radio button model.

So then we can evaluate which one the best to use.

Also I added the shortcut CTRL+E to toggle the modes + notifications to describe which mode it is in.

In curiosity do you use the Arduino IDE "extension"?

/Jannik
 
I tried to use Pure Data but as I couldn't use the mouse scroll for the sliders,
Also though it was hard to use (and to do special things), so I left that rather quickly.
Yup for my own synth I originally used Pure Data:
synthpd.png

but then I found "Ctrlr" https://ctrlr.org/ which is much easier to use (IMHO) and has a much "slicker" interface for putting down MIDI control widgets so my synth control panel (until I wire up all the knobs and switches) now looks like:

synthctrlr.png

In curiosity do you use the Arduino IDE "extension"?
Nope. I have a hard enough time building and cleaning errors on my local Arduino let alone trying to use a remote build. I guess it may have an advantage (depending on the server that it runs on) that it probably builds quicker than my ancient old laptop!

Oh one question I had for you - I know that in the C'tors of each class you generate you put the stuff to do all the AudioConnection's generated automatically but is there anyway in the web editing interface that I can say "and add this code to the c'tor". As your example shows what you end up with otherwise is adding a begin() method to almost every class to do the initialisation stuff that you probably would have done in the c'tor anyway. The only downside of using a separate begin() is ensuring that every one of them gets called after the class has been instantiated. There's a possibility of some code path leading to calls to other member functions before the being() has initialised them. Obviously the point of constructors is that you *know* that initialisation stuff will be done simply as a result of the class being instantiated.

Of course I can see why you might have both c'tor and a begin()/init()/whatever when there is maybe other stuff that can't happen as early as the time of construction.
 
I have now added a new node-type "ConstructorCode"
that can be used to add custom code at the end of the constructor.

Have also made it more clear when defining the main file:
WorkspaceEditForm.png
so that the "main" file can be given a name that's only used in the "Tool"
and to make it more strict/easier when exporting.


The Arduino IDE "extension" runs a local server inside the Arduino IDE

and is mostly intended to make
the transfer of the exported files
a lot easier and faster.

all you have to do is to follow the instructions on
https://github.com/manicken/arduinoAPIwebserver


otherwise export to zip can also be used
 
I have now added so that the exported zip file can now be imported again.

only use the Import-"JSON LoadFromFile"
and select the previous exported zip file.


Then the "Tool" looks if the "GUI_TOOL.json"
exists in that zip and then extracts and loads that file.
 
Jannik,

I see from Github that you have just been making changes to the web based audio design tool so perhaps I have just caught it "mid update" but something odd has started to happen. To further understand how to structure things I simply wanted to use "Menu-Examples-Manicken Poly Synth" to load your main example then "Export-Class based to Zip" to take a copy and just build it in the Arduino IDE on my machine but it threw build errors and that's because the generated files are full of...
Undef.pngMany/most of the generated lines have "undefined" at the start??

Now it's true I was playing about with many of the settings under "Settings" at the right so maybe I have set some silly option somewhere? But if it's that I wonder if there should be some kind of "reset all settings to default" to get it back to a known state where export will work?

Or maybe this is something being caused by something you recently changed?

Cliff
 
OK fixed now

It was because I had also added a new setting CodeIndentations @ Arduino-Export-"export code indentations"

I first named it CodeIncrementSpaces and then changed it into CodeIndentations
and forgot to change the name in the export code

(dynamic variable names one of javascript best features and also sometimes not so good)


The new setting is used to adjust the line Indentations used when exporting as class
and is also used in the ace code editor

Was thinking of having a separate setting in the editor but the export would then be messy with two types of Indentations.

/Jannik
 
Now there is a settings restore function
at the "Main menu"
ResetSettings.png

it works by saving the design without the settings
then it reloads the window
and in that process the default hardcoded values are used.

I have now made a change so that when a setting is changed
it automatically saves the project.
 
have now added force scroll by mouse wheel in gui run mode
press ctrl to activate the scroll function temporarily
can be combined with the shift key to scroll horizontally

also there are two new settings @ Workspace-Other:
Keyboard scroll speed
Gui run forced scroll speed
 
Jannik,

Can I make a feature request? It's possible to change the USB name of your synth if you do:

https://github.com/wrightflyer/Synth/blob/master/name.c

But as the comment I inherited from the thread where Paul first showed this says:
Code:
// To give your project a unique name, this code must be
// placed into a .c file (its own tab).  It can not be in
// a .cpp file or your main sketch (the .ino file).
Clearly this is about extern linkage and avoiding name mangling as the global generated in the file will over-ride a weak reference that is otherwise used. So would it be possible for the design tool to cope with a tab like this? At present if you try to name a tab with a name such as "name.c" it rejects this as it either needs to be a valid C++ class name or it has to be tagged as the the "Main File" but when you do that the naming possibilities are just .ino or .cpp (and anyway this is not the "main file"). So can you have a rename option to create "C file" that would then also allow the name to include a "." (like main.cpp does at present).

EDIT: ah ha - I see I can have more than one tab marked as "main file" (so it doesn't do the class wrapping) so it really is just about allowing a .c extension alongside the other .cpp and .ino options when a tab is marked as "main". (perhaps "main" should be "non Class" or something ?)
 
Last edited:
It should not be possible to have multiple main tabs, if so its a bug.
There is a node called code file, i have used it to include the mixer<n> in a file called theMixer.h
That node can be used to include any text based content to the project.
I can off course add new functionality so that a tab can be used/exported as a ordinary c file.

/Jannik
 
I just realised that the "CodeFile" that is used to include themixer.h in your example could be used to put a whole file inside one of the tabs so I can use that for name.c that I need. Only just realised this now. So don't need special handling after all.
 
yes and it don't matter in which tab it's placed, as every "CodeFile" is handled first in the export order.

Note.
when a "CodeFile" is placed in a tab that "CodeFile"-name is automatically added to the include list
of that exported "tab"-file

In the current version there is no filter for not including .c or .cpp files

but I have now updated the Tool so files ending with .c of .cpp are not put in the include list.

And to override that a "IncludeDef" node can be used
as the filter only affects "CodeFile" nodes

/Jannik
 
I have a idea

What do your think about using?

https://github.com/rogerwang/node-webkit

Building a Cross-platform Desktop App with NW.js
https://www.sitepoint.com/cross-platform-desktop-app-nw-js/

So that the tool is not restricted by local storage limits
And CORS (Cross-origin resource sharing)

This mean that the "Tool" would run like a standalone program and such have the possibility to update itself or at least parts of itself, for example the node definition list
And the embedded documentation.

I will try the web-kit and see how it works, and will post the results.

/Jannik
 
What do your think about using?
If you are asking me you are probably asking the wrong person. I'm a happy user of the utility but have no idea how it actually does its "magic" in the background. CSS/Javascript/etc are an alien world to me ;-)

But I love the result - I'm getting pretty close to having a full port of my synth (now with poly voices) working (well compiling anyway). I may have gone a bit over-board with the splitting into separate classes...

SynthClasses.jpg
 
One of the benefits of the current in-the-browser approach is that the user doesn't need to download or install anything. They just open the link in the browser. That's a really nice feature. I wouldn't give that up lightly.

Chip
 
I was just asking generally to everyone that reads this thread.

JavaScript and CSS
was also alien to me before I started making changes to this Tool,
I had no idea how the JavaScript code was working together, mostly because I started coding in notepad++

And then I moved to VScode as there is code completion
and somewhat code ref. search that makes it easier to follow the code.
Also the dynamic way that variables is created was very alien as it was hard to follow the structure,
I have now made some changes to that, so VScode is showing the structure of those parts.

I think its better to overuse the classing to make a structure that is easy to follow and maintain.

As a beginner its easy to just split the code into many .h files,
I also did it back then about 15 years ago when I programmed my first microcontroller projects (school class).
As before that it was alien to even program a pic-micro with a hex. (But had done some simple basic coding in my c64 at least)
 
chipaudette
I agree with you,
And off course the online webpage should also exist,
As it would be sharing the same code as the offline version, but the offline version can have extra features that is not possible while running in the browser sandbox environment.
 
And also the online version is easier to update.

Mostly I wanted this for storing big source files like wavetables, and to embed images for the GUI design part.
But there is a alternative to local storage, that is called indexedDB that have a lot higher storage limits, that can be used to embed big files into the Project.

Another plan for the offline version could be to create
something more like a complete standalone IDE that maybe uses the new arduino-cli, but that will not be something that happens in the near future I think.
 
To conclude the tests with NW.js

(no install needed, just a zipfile that is downloaded and extracted)

It's basically a Chromium browser with node.js embedded
but without standard browser features like searchbar, tabs etc.

It can even browse to any webpage (needs to be executed from the command-line)

And to ship some web-based program like the "Tool" as a standalone
All that's needed is to copy all files of the "Tool" into the same folder as the NW.js exe
and to create a manifest "package.js" that defines which file that is the root file (for example index.html)
Then the "program" is just started with the NW.js exe

The total size for that is ~333MiBi (350MB) 125MiBi packed.

I think it's good to know (for future tools/programs).
 
I have now changed the look and function of the workspace tabs
by using the new Node-Red code

So now the tabs can just be moved by drag/drop
instead of using move workspace buttons for a cleaner toolbar

The tab bar is also now scrollable (by two buttons) or mouse wheel scroll

The tab-widths is automatically adjusted based on the text widths.

That means we can now have long tab-names and many tabs.
NewTabs.png
 
Latest update includes a fix to the settings-tab
problem:
the settings that was saved was not loaded into the settings-tab, instead the default settings was always showing.
is now fixed (it was because I changed some code structure) I have now added a comment so it don't get
changed by mistake again.

update also includes a experimental "project"-tree-viewer:
ExperimentalProjectTreeViewer.png

the functions that can be used are:

* switching tabs (by clicking the "magnifying glass") on the "tab"-item

* locate a node (by clicking the "magnifying glass")

* select node don't work

* double click node in the tree opens the edit-form
(but also switches the tab, that can be disabled by unchecking setting "Right Sidebar"-"Auto switch to info-tab when selecting/deselecting node(s).")
 
Back
Top