Ethernet audio library

P

palmerr

Well-known member
I'm reworking my 2019 Ethernet Audio Library (old Ethernet Audio Library thread and Library) to take advantage of T4.1 native ethernet and to align the UDP packet protocol with what's being used by other products.

While there are a large number of network audio protocols to choose from (Wikipedia), many require special network hardware and others are proprietary. Of the remainder, the ones most common in pro audio equipment (Dante, Milan and Ravenna) are overkill for our purposes or have limited implementation information available to non-member organisations.

As I use Voicemeeter (donation-ware) as a desktop audio mixer with commercial audio interfaces, I have chosen its open network protocol (VBAN) as the basis for this revision. The Voicemeeter crew also provide apps for IOS and Android devices that will receive VBAN audio.

The new version:
  • 2, 4 and 8 channel input and output objects feeding to and from a single control_ethernet object.
  • Elastic buffers and dropped packet/overflow correction to resolve small master clock and block timing differences between hosts.
  • VBAN protocol compliance to allow multi-channel audio to and from desktop PCs and mobile devices.
  • Out of the box integration with Voicemeeter on the desktop and IOS/Android 'Receptor' apps.
So far
  • At the network layer I can receive up to 32 channels of audio packets from Voicemeeter on an isolated 100Mb network, using QNEthernet and Khoi Hoang's https://github.com/khoih-prog/AsyncUDP_Teensy41 library with out any dropped packets.
  • Wired into my home network, performance is not nearly as good. Whether this is because of some complexity with forward error correction on the WiFi side, or just fighting with other traffic I haven't yet determined.
  • Incoming packets are queued for the input object to unpack and transform into audio buffers.
Challenges ahead
  • Unpacking variable length incoming packets into timely audio streams.
  • Adequately buffering incoming audio streams while maintaining low latency.
  • Error correction for dropped packets and small differences in master clock rates.
  • Output streams which will be simpler as packet transmission is synchronous with Teensy audio update( ) cycles.

Current
  • Input packet transformation into Teensy audio. This is made somewhat more difficult as Voicemeeter can dynamically change a stream's sample rate and number of channels, and the number of samples per packet and packet interval varies with these parameters.
I'll update this head post with progress, with discussion following.
 
Last edited:
With asynchronous incoming packets I need to ensure the packet queue is not corrupted by interrupt-driven events occurring during update() cycles.

Is cli( ) / sei( ) the best approach, or is there a less global interrupt that can be temporarily blocked while manipulating queue pointers?
 
My suggestion is not to use the AsyncUDP_Teensy41 library plus your own buffering because the QNEthernet library already provides configurable buffering. This is, in fact, one of the reasons I don’t recommend that “async” library; when buffering is eventually required, the code becomes similar and now you have the bloat of another library. Note that it doesn’t actually use QNEthernet, just the included lwIP stack plus initialization routines. It also isn’t faster, more efficient, or asynchronous. Nor does it use interrupts.

The QNEthernet library buffering is transparent. To use it, simply create the EthernetUDP object(s) using the constructor that takes a size. For example: EthernetUDP udp{10}; for a queue size of 10. You can even resize the buffer at runtime via the setReceiveQueueSize(size) function.

If I had to guess, and please correct me if I'm wrong, you're receiving a UDP data event, adding to the queue, and in some other function you're reading data packets off that queue. You could do this all in one call with QNEthernet: the standard udp.parsePacket() API. That "async" library isn't faster, it's just a rearrangement, a "push" vs. "pull" style, but becomes the same once you do your own buffering/queueing. In other words, wherever you check/read from the queue, you'd just call that udp.parsePacket() function instead. Then there's a lot less code and setup.

Another point is that because that "async" library uses the QNEthernet driver calls under the covers, and because everything happens between main loop calls, not in interrupts, the packets don't actually come in asynchronously. They arrive in exactly the same way as using the Aruduino-style API, just in a different place.

All this also means that whatever interrupts you have going on won't affect any of the Ethernet stuff, as long as you're not doing Ethernet library operations within an ISR.

I have a question: Could you explain which interrupt events need to operate with the queued UDP data, either for writing or for reading? (Keeping in mind that that "async UDP" library doesn't use interrupts.)
 
Last edited:
Shawn,

Thanks for the info.

I had simply assumed that AsyncUDP was directly interrupt driven, rather than the lambda function polling for packets. So I assumed that I was going to have to deal with queue management from both update() and an interrupt driven environment.

QNEthernet's buffering seems to do what I require, so I should be able to drive everything synchronously with update() cycles.

At first glance, I think I will still need to queue any received UDP frames that I need to process further:
  • I need to have access to at least two consecutive frames for any form of dropped frame correction.
  • UDP frames may be split across multiple audio buffers as the protocol I'm using allows any number of samples per frame. My desktop VBAN test application, Voicemeeter, produces packets with 89, 119, 179 and 103 samples per frame for 8, 6, 4, 2 channels).

I'll read more deeply into both QNEthernet and AsyncUDP before proceeding further, but I think there's definitely a shift away from AsyncUDP in the wind!

Richard
 
Last edited:
I built and use the QNEthernet library for my own uses and use it in the field for streaming buffered pixel and controls data, and serving web pages for configuration. But it can do a lot more. I’m looking forward to your experiences using it for audio data. Make sure to check out the README.
 
Here's a tip: To access a packet's data, you don't need to copy it into a local buffer. You can simply call data() on the UDP object to access a pointer to the data, and size() to get the data size. That will save some memory and time.
 
Thanks Shawn,

I have read the readme and probed lightly into the code. Queued packets will be very useful for managing any incoming packet overruns at the UDP layer, meaning I can do all my processing with synchronous calls.

There can be several incoming streams of audio packets, possibly from different hosts (and at different packet intervals). I need to be able to identify and assign each packet to an appropriate audio stream and then process it as required, not necessarily in the order the packets arrived.

So, I think I'm still going to have to have a separate queuing arrangement. Even peeking further down the queue won't completely solve the problem, as the next packet I have to process for a particular stream may be anywhere in the queue. It would then need to be read() and unlinked from the middle of the queue.

I also need to be able to check packet sequence numbers (inside the next VBAN packet related to this stream) for error correction before processing. The simplest form of dropped block correction is to peek() the next two packets related to this audio stream. If there's a dropped block the next data packet is peek()ed rather than read() and its packet sequence number incremented, so that it gets processed twice.

Not so difficult to do with a linked list, but messy with a pure queue where the middle is invisible!

I'm not familiar enough with the std:: queue and vector template classes to be able to fathom exactly how this might be done!

Thanks anyway for QNEthernet.

Richard
 
Thinking aloud here… I wonder if it would be useful if I gave access to the internal queue, or if what you suggest, having a separate queue in your program, is the way to go. “UDP layer” and “application layer” being separate, etc.

Or maybe I should just add a callback for UDP packets? I’m not on board with that quite yet, though.
 
Shawn,

I've been thinking about the same issue.

If it were just peeking deeper into the queue, then access to the internals would be useful.

For my application, with multiple, asynchronous UDP packet streams, a linked list is a better data structure as I might have to push stuff back onto the queue for error correction.

For others, being able to peek further down the queue as well as have the count that's already available may well be advantageous.

When I've finished the first working draft of the 'two-queue' version, I'll review the queue management issues that have needed to be addressed and will be able to give a fuller answer.
 
The internal queue is a circular buffer (a vector) because then there’s no extra allocation. With a linked list, I’d have to use a pool of pre-allocated objects, and I think the implementation would be a little more complex. It sounds like doing the linked list, if that’s the need, in the application might be better.

I like the idea of possibly making some improvement to the library that helps your use case and others like it. I just don’t know what that is yet. :)
 
Shawn,

I've run into a snag... getting this error when calling QNEthernet routines from within a software interrupt (Teensy Audio Library uses them for its regular updating).

c:\Users\xyzzy.DESKTOP-J4ESB3N\Documents\arduino\libraries\QNEthernet\src\netif\ethernet.c:89:ethernet_input()
Assertion "Function called from interrupt context" failed at line 145 in c:\Users\xyzzy.DESKTOP-J4ESB3N\Documents\arduino\libraries\QNEthernet\src\qnethernet_hal.cpp

The error occurs with both
EthernetUDP udp{10};
or
EthernetUDP udp;

and startup code copied out of the BroadcastChat example.

The only calls I'm using within the software interrupt are
udp.parsepacket()
udp.size()
udp.data()
and udp.remoteIP()

My code is quite complex, and the error doesn't occur in code that doesn't use the software interrupt. Any clues on how I should go about tracing the error?

I'm creating a cut-back version of the code to demonstrate the issue.
 
The library isn’t designed to be called from within interrupts. The alternative is to set a (volatile) flag in the interrupt and then check that flag to do the operation from your main loop somewhere (and then clear the flag), i.e. not in an interrupt context.

Allowing the library to be thread-safe and interrupt-safe introduces a whole bunch of messy.
 
Last edited:
Better yet, in the interrupt, append the data you need to send to a concurrent-safe or “mutexed” queue/buffer and then in your main loop (or in a function called from your main loop), send the data you need from the queue.
 
OK,

It's not good practice to call Teensy audio library update functions from the main loop as they need to be sequentially called when a new audio buffer is required for processing - hence the regular software interrupt. I'll have a think about other approaches!
 
I've used EventResponder to good effect for decoupling the audio update loop from less deterministic data sources or sinks. In my case it's filesystems, which can be a bit slow but always respond eventually. For Ethernet audio the situation is clearly different, but at least you can get a trigger from the audio update into your code in a timely manner, and vice versa.

EventResponder typically hooks into the yield() processing, which isn't without its issues, but it may save a bit of wheel-reinvention if you can make it work for you.
 
palmerr said:
It's not good practice to call Teensy audio library update functions from the main loop as they need to be sequentially called when a new audio buffer is required for processing - hence the regular software interrupt. I'll have a think about other approaches!
Click to expand...
Just to be clear, I wasn’t suggesting doing audio updating from the main loop, just sending things over Ethernet from the main loop, where the things you’re sending are buffered in the interrupt.

This is almost equivalent to hooking into yield() via EventResponder because those hooks are called basically every time the main loop ends, when yield() gets called internally. And, of course, every time yield() gets called elsewhere.
 
I can try that with some semaphores in the update() code to provide protection against both routines trying to manipulate queue at the same time. Something in the main loop isn't pretty, but EventResponder looks promising to resolve that. Extra yields could help with slow main loop code. Now all I have to do is read that thread in detail and then work out how to hook into Event responder - perhaps I'll wait until that other thread comes to a conclusion!

Later: Yes, that worked.

BTW, thanks for the new code this week.
 
Last edited:
See QNEthernet.cpp for code that uses EventResponder, as an example. You may have discovered that you need to call triggerEvent() on the argument. That makes it get called again.

Note that using EventResponder with yield() is almost equivalent to just calling a function each time through the main program loop(), so personally, I usually just do that in my applications, and is also the approach I recommend because it’s simpler. The only difference is that the function registered to the EventResponder is also called wherever yield() is called elsewhere, not just after each program loop(), and this isn’t always critical. Unless it is. :)

It’s specifically handy for libraries that wish to call something regularly, like in QNEthernet.
 
Thanks - I'll ponder the options. Standard Teensy Audio practice is to hide the regular loop calls. While testing, I'm using an explicit call in the main loop to avoid yet another complexity!

On a different note, a possibly dumb question: which is the better way to block AudioStream update() calls while I'm manipulating queues shared by AudioStream and other code?

cli() / sei()

or

__disable_irq() / __enable_irq()
 
Last edited:
Those are the same. sei() is defined to be __enable_irq() and cli() is defined to be __disable_irq(). Also interrupts() and noInterrupts() are defined to be those too.
 
You must log in or register to reply here.
Back
Top