Open Sound Control (OSC) Teensy Audio Library Implementation

I think it's best if you also check for empty strings @
OSCAudioBase::createObject
So that an object is not created without any name or do not contain any whitespace, you could also get all in and trim any whitespace.
Or if a whitespace exists inside a name it automatically is translated into a _ (underline)
I don't think there's anything I can usefully do to refuse to create an object with a NULL, empty or duplicate name! It's always possible to build them into a piece of code... The renameObject() function uses find(), which will always return the first match, so you can gradually rename them if there are several, and it doesn't allow you to rename to the name of an existing object ... hmm ... though come to think of it, there's no way to tell the type of an object from its name ... I should add that for sure. If I can figure out how.

I don't believe OSC really cares about spaces, they have no syntactic significance, so I'd rather leave them in if a user wants them.

EDIT: our posts crossed - but I still think we leave it to the user to make a mess if they want!

New version pushed up, with renaming implemented. Use /teensy1/dynamic/rename<"oldName"><"newName">. There's a Python example, too.
 
but I still think we leave it to the user to make a mess if they want![/I]

yes but I believe that the whitespace trim is a good thing
because if someone accidentally have any spaces around the names
then they are automatically removed,

as it's really hard to pinpoint any errors occurring when doing so


and by the way right now the users are already used to that any spaces are replaced by _ in the official tool
and it's a easy function


here is the renameObject
with "empty name check
Code:
/**
 *    Rename an [OSC]AudioStream or Connection object.
 *  This could be an /audio function, but that would pollute the 
 *    audio functions' name spaces, so we make it a /dynamic
 *  function instead.
 */
void OSCAudioBase::renameObject(OSCMessage& msg, int addressOffset)
{
    char oldName[50];
    char *newName;
    newName = (char*) malloc(50);
    char* newNameOrigin = newName; // so that free can be used later (as the trim function changes the start address)

    OSCAudioBase* pVictim;
    
    msg.getString(1,newName,50);
    
    newName = trim(newName);
    replaceWhiteSpace(newName, '_');

    if (strlen(newName) == 0) Serial.println("empty name"); // don't allow any kind of "empty" name
    else
    {
        pVictim = OSCAudioBase::find(newName);
        if (NULL == pVictim) // we're not duplicating the name of another object: good
        {
            msg.getString(0,oldName,50);
            
            pVictim = OSCAudioBase::find(oldName);
            if (NULL != pVictim)
            {
                pVictim->setName(newName);
            }
        }
    }
    free(newNameOrigin);
}

also I wonder what happens with
char oldName[50];
when the function exits,
do it get automatically cleaned?
 
I have now updated the scripts in the Tool
so that the following events are sent to the Teensy as OSC messages:

Audio Object Added
Audio Object Renamed
Audio Object Removed
AudioConnection Added
AudioConnection Removed

note.
when a connected AudioObject is removed in the tool
two actions are sent (in that order)
AudioConnection Removed
and
Audio Object Removed

but It looks like the Web Serial API need some time between each send
so only the first
AudioConnection Removed
is sent


I will look how this can be fixed
That is also why I choose to send certain messages as bundles

Maybe I have to use bundles to this to combine
AudioConnection Removed
Audio Object Removed
when a AudioObject is removed


The AudioConnection Added
do contain a bundle to both send "create AC" and "connect"
 
yes but I believe that the whitespace trim is a good thing
because if someone accidentally have any spaces around the names
then they are automatically removed,

as it's really hard to pinpoint any errors occurring when doing so


and by the way right now the users are already used to that any spaces are replaced by _ in the official tool
and it's a easy function


here is the renameObject
with "empty name check
Code:
/**
 *    Rename an [OSC]AudioStream or Connection object.
 *  This could be an /audio function, but that would pollute the 
 *    audio functions' name spaces, so we make it a /dynamic
 *  function instead.
 */
void OSCAudioBase::renameObject(OSCMessage& msg, int addressOffset)
{
    char oldName[50];
    char *newName;
    newName = (char*) malloc(50);
    char* newNameOrigin = newName; // so that free can be used later (as the trim function changes the start address)

    OSCAudioBase* pVictim;
    
    msg.getString(1,newName,50);
    
    newName = trim(newName);
    replaceWhiteSpace(newName, '_');

    if (strlen(newName) == 0) Serial.println("empty name"); // don't allow any kind of "empty" name
    else
    {
        pVictim = OSCAudioBase::find(newName);
        if (NULL == pVictim) // we're not duplicating the name of another object: good
        {
            msg.getString(0,oldName,50);
            
            pVictim = OSCAudioBase::find(oldName);
            if (NULL != pVictim)
            {
                pVictim->setName(newName);
            }
        }
    }
    free(newNameOrigin);
}

also I wonder what happens with
char oldName[50];
when the function exits,
do it get automatically cleaned?
I stand corrected - there's a bunch of characters not allowed in OSC Methods or Containers: <space>#*,/?[]{}. I'll put in a function to sanitise incoming strings, to be applied as needed.

No, C and C++ don't automatically do anything (much)! Any memory allocated from the heap using malloc() has to have a corresponding free(), or you end up with a memory leak. it's an irritating omission in CNMAT / OSC that though there's a getBlobLength() there's no corresponding getStringLength(), so we have no way of allocating just enough space for string parameters before copying them. My unguarded 50-char arrays are poor, and I need to do something about them...

I have now updated the scripts in the Tool
so that the following events are sent to the Teensy as OSC messages:

Audio Object Added
Audio Object Renamed
Audio Object Removed
AudioConnection Added
AudioConnection Removed

note.
when a connected AudioObject is removed in the tool
two actions are sent (in that order)
AudioConnection Removed
and
Audio Object Removed

but It looks like the Web Serial API need some time between each send
so only the first
AudioConnection Removed
is sent


I will look how this can be fixed
That is also why I choose to send certain messages as bundles

Maybe I have to use bundles to this to combine
AudioConnection Removed
Audio Object Removed
when a AudioObject is removed


The AudioConnection Added
do contain a bundle to both send "create AC" and "connect"
Sounds good - I assume you've upgraded the .ino testbed to accept bundles? I'll take a look soon.
 
@h4yn0nnym0u5e

You forgot to add Serial.println("blank name");
to OSCAudioBase::createConnection
could be good for debugging purposes

but believe all these need to be replaced by some global "error code" handling stuff anyway
so there is the same functionality as the OSC lib uses
ex.
Code:
bool hasError();
    
OSCErrorCode getError();

and then the error output handling can be in the main code if wanted.
here are some "errors" that I can think about:
OSC_OK
OSC_NO_TARGET
OSC_TARGET_NO_FUNCTION
OSC_FUNCTION_PARAMETER_MISMATCH
OSC_TARGET_BLANK_NAME
this also means that if a error occurs then a OSC message can easly be sent back to
the "sender", so that we don't need the debug output for that.

also it could be a good thing in the OSCAudioTesting.ino
to use the OSCMessage/OSCBundle empty function to free any data


to get the string length we can use this as a workaround,
in the OSCAudioBase class add the following
(based on the OSCMessage::getBlobLength and OSCData::getString(char *) code):

Code:
int getStringLength(OSCData* datum)
{
    if (type == 's'){
        return bytes;
    } else {
    #ifndef ESPxx
        return (int)NULL;
    #else
        return -1;
    #endif
    }
}
int getStringLength(OSCMessage *msg, int position)
{
  OSCData * datum = msg->getOSCData(position);
  if (!msg->hasError()){
    return getStringLength(datum);
  } else {
    #ifndef ESPxx
        return 0;
    #else
        return -1;
    #endif
  }
}

so then you don't need to allocate 50 bytes
and the string can be any size
 
I'm now simplifying the scripting for the Different Dynamic handling events
as now the structure is known,
so they are all gonna be simplified to only
one script which only contains one line
Code:
OSC.SendAsSlipToSerial(data);

that line is essential just to define where,
over which "physical" layer (Serial,MIDI sysex,WebSockets,HTML post)
the data is gonna be sent

I will post again when it's done.


edit.
At the final stage that script line is gonna be replaced by just a combobox selector
to define the "physical" layer
 
As you say, we really need (as in, I really should do...) some OSC replies to pick up when errors have occurred. For now, if you try to create something and it doesn't succeed, you can see it (from a sketch) using something like my listObjects() function.

Don't think we need to use the OSCMessage/OSCBundle empty() function in OSCAudioTesting.ino to free data, as it's destroyed and a new one is instantiated every time loop() executes. Pretty sure the destructor frees the allocated memory - we'll soon know if not.

Might edit the OSC library and do a PR for getStringLength(), since we will soon have Adrian's attention.

Looking forward to your GUI updates, they sound cool and will save me some tinkering!
 
Have just pushed a set of updates up. You'll need the latest OSCAudio and a new dynamic audio library, as some audio objects refused to compile for Teensy 4.x. Not a problem for most people, but being able to use OSC to create anything does tend to reveal the flaws.

I'm assuming that everyone currently interested in this has a Teensy 4.x to test on. I've not yet ported the dynamic audio stuff to Teensy 3.x, and would rather not do that right now. Plus, OSCAudioTesting.ino in a dynamic environment is coming out at over 320kB (code+data), so I'm not 100% sure if this is going to work for Teensy 3.x anyway - at least not with the GUI.
 
Now it's done.
I did also remove the last simple line script.
It's all replaced by this
OSCsettings.png

It's only the Web Serial API that's implemented.


I did however have some problems
from a scratch design based on the example
basicDesign.png

here I can add all different kind of objects and connect them together

but as soon as I remove one object something breaks
and I'm getting INVALID_OSC

by error checking at
Code:
if (!bundle.hasError())
{
      .........
      .........

}else {
      OSCErrorCode error = bundle.getError();
      HWSERIAL.print("bundle error"); 
      OSCMessage errorMsg("/error/bundle");
      
      if (error == OSCErrorCode::BUFFER_FULL)
        HWSERIAL.println("BUFFER_FULL");
      else if (error == OSCErrorCode::INVALID_OSC)
        HWSERIAL.println("INVALID_OSC");
      else if (error == OSCErrorCode::ALLOCFAILED)
        HWSERIAL.println("ALLOCFAILED");
      else if (error == OSCErrorCode::INDEX_OUT_OF_BOUNDS)
        HWSERIAL.println("INDEX_OUT_OF_BOUNDS");

  }

Enjoy!
 
That's great ... nearly! I'm having huge problems:
  • I can't edit a slider properly: when I click OK nothing happens, though Apply then Cancel can work
  • If I try to delete elements of the existing design, then refresh (F5), everything comes back exactly as before
  • Sometimes if I start dragging an object, I can't drop it
  • If I delete something, it doesn't disappear immediately, but only when I click elsewhere
Having said that, I have just about managed to use the GUI to create a new waveform, link it to an existing mixer, start it sounding, and set up a slider to vary its frequency. So all the elements are really close to coming together.

I too had problems with spurious bundle / message errors: the latest OSCAudioTesting.ino seems to get round them for now. It looked like I was getting SLIP packets starting with a 0 character, which I put down to my Python test code, but maybe it's more fundamental than that. The sketch only deals with bundles of messages for now, and ignores the timetag.
 
There is some problems with the project-tree functionality
Don't know why, as it clearly worked before,
I need to investigate this later when there is time.

I have now totally disabled that feature.
As your problems could come from there.

I did however not had any big problems as you describe.

I also disabled a lot of debug printing,

so if the problem still exist
You can check in the browser "developer tools" F12
or at the "menu in chrome"-"more tools"-"developer tools"
then check on the Console tab for errors


And I forgot to mention that there is some new functions available at the OSC namespace
that makes it much easier to put together messages and bundles
without all that JSON structures
I really like JSON but in this case it just take up so much space and makes it hard to structure up in loops.

here is all the event and helper functions
hope you can learn something from them
Code:
function CreateMessageData(address, valueTypes, ...values)
{
    return osc.writePacket(CreatePacket(address, valueTypes, ...values));
}

function CreateBundleData(bundle)
{
    return osc.writeBundle(bundle)
}

function CreatePacket(address, valueTypes, ...values)
{
    var minLength = valueTypes.length;
    if (minLength > values.length)
    {
        minLength = values.length;
        AddLineToLog("(ERROR) @ OSC.CreatePacket() valueTypes length mismatch count of values<br>nbsp;nbsp;some parameters are trimmed", "#FF0000", "#FFF0F0");
    }

    var packet = {address:address, args: []};

    for (var i = 0; i < minLength; i++)
    {
        packet.args.push({type:valueTypes[i], value:values[i]})
    }
    return packet;
}

function CreateBundle(timeDelaySeconds)
{
    if (timeDelaySeconds == undefined) timeDelaySeconds = 0;

    return {timeTag: osc.timeTag(timeDelaySeconds),packets:[]};
}

function NodeAdded(node)
{
    if (node._def.nonObject != undefined) return; // don't care about non audio objects

    var addr = RED.OSC.settings.RootAddress + "/dynamic/createObject*";

    SendData(CreateMessageData(addr,"ss", node.type, node.name));
    
    if (RED.OSC.settings.ShowOutputDebug == true)
        AddLineToLog("added node (" + node.type + ") " + node.name);
}

function NodeRenamed(node, oldName, newName)
{
    if (node._def.nonObject != undefined) return; // don't care about non audio objects

    var addr = RED.OSC.settings.RootAddress + "/dynamic/ren*";

    SendData(CreateMessageData(addr,"ss", oldName, newName));

    if (RED.OSC.settings.ShowOutputDebug == true)
        AddLineToLog("renamed node from " + oldName + " to " + newName);
}

function NodeRemoved(node, links)
{
    if (node._def.nonObject != undefined) return; // don't care about non audio objects

    var addr = RED.OSC.settings.RootAddress + "/dynamic/destroy*";

    var bundle = CreateBundle();

    for (var i = 0; i < links.length; i++)
   {
        var link = links[i];

        if (RED.OSC.settings.ShowOutputDebug == true)
            AddLineToLog("removed link " + GetLinkDebugName(link));

        var linkName = GetLinkName(link);

        bundle.packets.push(CreatePacket(addr, "s", linkName));
    }
    bundle.packets.push(CreatePacket(addr, "s", node.name));

    SendData(CreateBundleData(bundle));

    if (RED.OSC.settings.ShowOutputDebug == true)
        AddLineToLog("removed node " + node.name);
}

function LinkAdded(link) {
    var connName = GetLinkName(link);

    link.name = connName;

    var addLinkAddr = RED.OSC.settings.RootAddress + "/dynamic/createConn*";
    var connectLinkAddr = RED.OSC.settings.RootAddress + "/audio/" + connName + "/connect*";

    var bundle = OSC.CreateBundle();

    bundle.packets.push(CreatePacket(addLinkAddr, "s", connName));
    bundle.packets.push(CreatePacket(connectLinkAddr, "sisi", link.source.name, link.sourcePort, link.target.name, link.targetPort));

    SendData(CreateBundleData(bundle));

    if (RED.OSC.settings.ShowOutputDebug == true)
        AddLineToLog("added link " + GetLinkDebugName(link));
}

function GetLinkName(link)
{
    if (link.name != undefined)
        return link.name;
    else
        return link.source.name + link.sourcePort + link.target.name + link.targetPort;
}

function GetLinkDebugName(link)
{
    return "(" + link.source.name + ", " + link.sourcePort + ", " + link.target.name + ", " + link.targetPort + ")";
}

function LinkRemoved(link)
{
    var addr = RED.OSC.settings.RootAddress + "/dynamic/destroy*";
    
    var linkName = GetLinkName(link);
    SendData(CreateMessageData(addr,"s", linkName));

    if (RED.OSC.settings.ShowOutputDebug == true)
        AddLineToLog("removed link " + GetLinkDebugName(link));
}

these are the exposed functions
Code:
SendData // uses the encoding and transport layer settings
SendRawToSerial
SendAsSlipToSerial
SendTextToSerial
CreateMessageData
GetSimpleOSCdata // annother name for CreateMessageData // keep this for backwards compability
CreatePacket
CreateBundle
CreateBundleData // simplifies osc.writeBundle(bundle)
AddLineToLog // simplifies usage of RED.bottombar.info.addLine(text);
 
Very quick post, I have to do domestic stuff this morning!

I deleted all the cookies and the GUI seemed to be working much better. Then I reloaded a saved file View attachment TeensyAudioControlDesign3JSON.txt (.JSON isn't valid as a forum attachment: remove the .txt and put the dot before the JSON!), and it broke again. There may be a clue for you in there, but don't spend too much time, as I've not really explored what's working now that wasn't before.
 
Last edited:
Had another chance to play, deleting existing data seems to have fixed things.

Is it possible to add the capability to "send current design to Teensy", i.e. re-send all existing / visible objects and connections? It will ignore duplicates, of course, but that's fine. And I need to do a corresponding "delete all dynamic objects", so a user can clear the target and load a design in.

Off out to lunch now...
 
Is it possible to add the capability to "send current design to Teensy", i.e. re-send all existing / visible objects and connections?
Funny thing I did just think about implementing that,
Think I will put it into the export menu, but maybe the dynamic stuff should have a separate menu?
It will then use one bundle to send everything:
1. AudioObjects.
2. AudioConnections

I'm not at the computer right now, believe I also need to do some domestic work first.
 
If you come up with a address for the clear design OSC message
Then I can also include (when I have time) that in the send design bundle
 
I have now implemented
"send current design to Teensy"
by reusing the standard export code structure.

I did use the following name for the clear design address
/teensy*/dynamic/clearAll*

available at new
OSC menu - Simple (only the design at the current tab)

I was able to import your design file
when changed file extension to JSON
so no problems here

But there is something wrong with my "settings system"
if a setting exists in the JSON then there will be an error
I did a dirty fix for that but need to do it more proper later.

In your design file (JSON) there still exists a Reference to NodeAddedScript
if that is removed

from:
Code:
"OSC": {
            "NodeAddedScript": "RED.bottombar.info.addLine(\"added node \" + node.name);\nRED.bottombar.info.addLine(\"\"+node.typename);"
        }

to:
Code:
"OSC": {}

then you should not get any error at all
 
OK, new commit made:
  • emptyAllObjects() added, accessed at /dynamic/e*, will remove all objects known to the OSCAudio system (but not those created without OSC support)
  • problem with methods which have no parameters fixed: used to cause crash, we can now disconnect connections, use enable() on SGTL5000, etc.
  • added example OSCAudioEmpty.ino sketch, which has no audio objects created by default, so good for testing the GUI++

Posts crossed again! Think I'll change to your name for clearAll, we don't want it to be executed accidentally...
...done
 
This is looking really good.

Slight problem with the OSC Export to Dynamic Audio Lib: the object types are sent with a lot of trailing spaces, so they don't match the names and never get constructed.

Code:
{"address":"/teensy*/dynamic/createObject*","args":[{"type":"s","value":"AudioSynthWavefo[COLOR="#FF0000"]rm               "}[/COLOR],{"type":"s","value":"waveform2"}]}
 
Last edited:
Slight problem with the OSC Export to Dynamic Audio Lib: the object types are sent with a lot of trailing spaces, so they don't match the names and never get constructed.

Have now fixed that,
(one slight problem when reusing old code).

Also did shorten down the "cmd":s
so they are now

Code:
/dynamic/cr*O*
/dynamic/cr*C*
/audio/connectionName/c*
so saving some space for bigger designs

maybe they could be even shorter?
Code:
/dyn*/cr*O*
/dyn*/cr*C*
/au*/connectionName/c*

+ fix some esthetic things in the dialog

edit.
maybe we don't need the full /teensy* either
just /t* should be enough
but that can easily be changed by the user,
both in the sketch and the Tool "OSC" Settings Tab
 
Is there a limitation on how long the addresses can be? Using shortened names as opposed to human readable ones?
 
Of what I can see in the code there is no limit to any part of the OSC "decoder" messages/bundles

But there is maybe a memory limit of the teensy

there is this huge design from
kd5rxt-mark
Total AudioObjects:433
Total AudioConnections: 640

the raw message size for that is
119556 bytes
and that is with short commands
with long commands
that is
134764 bytes

diff 15208 bytes
so maybe not to much to save there

also tested this huge design
https://forum.pjrc.com/threads/63668-Limits-of-the-GUI?highlight=gui+limits

but that is only
Total AudioObjects:300
Total AudioConnections: 513
RAW data (size 88460 bytes):


I also broke the code exports
by trying to "split" out functions to a external file
It's now back to a working state.

Also now it accepts plain/text as JSON file import
but it's recommended to keep the json file extension
for clarification
 
Excellent - all working for me here.

Radio silence yesterday was getting the 647b8d3 commit sorted to a reasonable level. We now have:
  • all commands get a reply, in the form of a bundle whose messages are all addressed to /reply
  • the number of messages in the bundle depends on the number of addresses "hit"; if your command address has /mixer* in it and you have mixerL and mixerR, you'll probably get two messages in the reply bundle
  • audio command reply messages are of the form ss<X>:
    • first s is the address of the command received
    • second s is the name of the object affected
    • <X> is the returned value, which may be of various types - for void functions it will be boolean T, otherwise it's likely to be f, i or something
  • dynamic command reply messages are of the form ssi:
    • first s is the address of the command received
    • second s depends on the command:
      • connect: objects and ports connected
      • create/destroy object or connection: name of object created/destroyed (may be different from requested due to the "sanitisation" function)
      • clear all: "ALL"
      • rename: old -> new names
    • i is an error return value - 0 to 3 are OK,NOT_FOUND,BLANK_NAME,DUPLICATE_NAME
I've updated OSCAudioEmpty.ino to implement this. Note that it is the programmer's responsibility to decide where the replies go to; it's entirely reasonable to create your preferred reply format based on the library-supplied bundle. We will have to agree on what the GUI++ expects. Hmm ... should we allow for some sort of message from the GUI saying "this is the GUI on this stream"? That could include an element to set the 64-bit time, too, so we can think about timed bundles.

Playing with this has shown up a problem with not recording the connection names: if you connect mixerL to i2s you might get a connection called mixerL0i2s0; if you then rename mixerL to mixer1, and delete the connection, the GUI tries to delete mixer10i2s0, which doesn't exist. So I think it is necessary to track connections in the GUI, after all.
 
Back
Top