Teensy 4.1 as 16ch 24b/48kHz TDM to Ethernet server PLUS OSC server?

Status
Not open for further replies.
C

celoranta

Member
Hi All.

I'd like to take the 48kHz/24b TDM8 outputs of two Cirrus 8-channel ADCs and serve them uncompressed via a TCP port on the Teensy. In theory, TDM8 serial data would be written to memory in a struct containing an incrementing 'sorting' value and a 1-16 'channel' value. Memory register will have a fixed size, and gracefully drop any packets older than "x" increments.

A client application will connect via the Teensy's IP address to request any available audio data. Any audio data in the register will be packetized and sent to the client, then dumped from the register.

The client-side app will include a configurable sample buffer, allowing the user to select a buffer size that produces least latency without audio drop-out. Received packets will be written to a playback register in the sequenced order and played back at 48kHz within the app.

All this would need to be built on top of/around a project like this one:

https://forum.pjrc.com/threads/56117-PoE-powered-OSC-controller-for-Reaper-DAW

Which would simultaneously transmit and receive OSC control data via the same Ethernet jack on separate UDP ports. OSC is latency-tolerant (or ought to be, as it is built on top of a standardized ethernet stack) and should never block audio processing tasks.

Still considering whether this would be possible on a single Teensy, or whether separate audio and OSC boards ought to serve preprocessed data to a third board acting as the network server. I'd think there are plenty of cycles on the Core to do both, but it may require extensive rewriting of the existing code which might be beyond my capabilities.

I don't really have the Microcontroller Dev chops for this project, but I'm a longtime follower of such projects and have deep background in electronic circuitry, digital audio, musical control protocols, programming, and ethernet... so I believe I'm game to try...

I welcome any and all comments from interested parties on whether Teensy is the right call here, and where to start. So far I've looked at the Teensy Audio project (which I believe is 44.1k/16b, so **no longer in the discussion.)

This project has also caught my attention:
https://forum.pjrc.com/threads/61942-Audio-bridge-to-Ethernet-is-it-possible?p=289856#post289856

Thanks to the creators of the linked projects for all their excellent work on the subject!

**Do note that 48k/24b/16ch is a hard-specified requirement, lest the conversation drift into "Why would you need THAT?"-land. I understand the arguments on both sides, but would rather not spend time on that discussion. (This project will interface with an extremely complex and extremely expensive existing audio/network system, which cannot be run at 44.1k/16b due to compatibility with existing hardware and software. The choice of sample rate and bit depth was made many months ago and doubled-down upon during system purchase/build.)
 
jonr said:
As much as possible, I'd stick to standard protocols. Like RTP.
Click to expand...

I've considered approaching this in two different ways:

1 - As you suggested, using protocols specifically tailored to real-time audio transmission. No need to invent the wheel, assuming the wheel exists. I suspect this RTP payload format would suit my needs, as it allows for 24bit audio with interleaved multi-channels:
https://datatracker.ietf.org/doc/html/rfc3190

2 - Direct serialization. Another developer who accomplished something similar suggested staying as close as possible to the incoming serial data, simply serializing it and passing it along. His app works well enough, so the suggestion stayed in the conversation. However, I'm dealing with two separate 8-channel TDM stream inputs, so some sort of processing would be required to keep the packets separate (or maybe this just means serializing the inputs separately and sending two serial streams... that actually makes sense.)

I'm open to discussion along either/both routes. If anyone out there has built upon the RDP stack with interleaved channels of audio, I'd be interested in hearing from you!
 
It is very likely the CPU and ethernet hardware can do it. My reference is having done this sort of thing long ago on SGI machines and Mac's with slower processors than the Teensy. We were doing 8 channels of audio and OSC with the SGI O2. In fact I did the first implementations of OSC on early Sony UNIX laptops and even slower SGI machines.

Your bit rate is 18.4Mbit/s which is shy of the 100Mbit/s of the ethernet PHY. There is a fair amount of packetization overhead but there should be bandwidth left over for OSC. Is there enough bus bandwidth and DMA channels internally for the ADC at that rate along with about twice that going out to ethernet? Also I have no experience using TCP for this - we always used UDP. We just didn't want to deal with the complexity of buffers and resending.

In the last projects I did of this sort we had 128 channels of audio for a microphone array and another for a speaker array and a few dozen others for a custom gesture controller.
We did this with low latency, no lost packets and UDP. The thing is we used FPGA's to do all the heavy lifting so there was no asynchronous buffering, caches and contention issues. We could get this many channels in and out of an old beefy Mac using a custom driver.
 
Another thought comes to mind: work on the protocol design and get this going on a regular computer with an audio interface first. This will give you an easy reference and benchmark baseline.
 
First off, I notice your name is “Adrian Freed,” which is (by no coincidence, I’m guessing) also the name of one half of the original OSC development team. Thanks very much for commenting on my little thread, Adrian.

If the teensy (or several teensys) is not up to the task, I’ve got xmos Xcore200 as my backup plan. Lots of parallel processing there, but much steeper learning curve and very expensive (and back ordered) dev boards.

I have no issue with UDP in concept, but my rig does include a switch hop between dumb switches, and I may be forced to upgrade to Cisco Sg350s/sg250s to keep the Dante component of my rig separated. Am I correct in remembering that UDP packets cannot cross a VLAN?
 
Oh dear. I’m afraid the complexity overhead of the OS’s out there put me at a real disadvantage here. I can break out my Swift if I must, but I turned to Teensy hoping to avoid all that…. I shall look into it. And thank you.
 
celoranta said:
First off, I notice your name is “Adrian Freed,” which is (by no coincidence, I’m guessing) also the name of one half of the original OSC development team. Thanks very much for commenting on my little thread, Adrian.

If the teensy (or several teensys) is not up to the task, I’ve got xmos Xcore200 as my backup plan. Lots of parallel processing there, but much steeper learning curve and very expensive (and back ordered) dev boards.

I have no issue with UDP in concept, but my rig does include a switch hop between dumb switches, and I may be forced to upgrade to Cisco Sg350s/sg250s to keep the Dante component of my rig separated. Am I correct in remembering that UDP packets cannot cross a VLAN?
Click to expand...

UDP works across VLAN's
Sometimes there are config. issues especially with broadcast packets.

It's hard to shape the traffic if there is a VLAN involved so we always did things with controlled LANs and known-good switches.
A lot depends on how much latency you can manage. If you can absorb latency you will need to buffer some on the Teensy. There is enough room but
it is good to confirm this with the RAM2 column you have probably seen here: https://www.pjrc.com/store/teensy41.html
 
Latency is not a huge issue for the OSC side (Can probably tolerate 25-50 ms?) The audio side, however, must be as close to real time as is possible. My teensy will also have the two optional flash chips installed, which may allow extra room for buffers?

My 25-50ms tolerance guess is based on the assumption that “MIDI CC”-style data (for filter sweeps, volume swells, pitch bends, damper pedals, etc) is not generally used in a way that demands the 10ms of latency required for post-fx monitoring by a performing musician.

It sounds like all roads point toward UDP, then…
 
adrianfreed said:
Another thought comes to mind: work on the protocol design and get this going on a regular computer with an audio interface first. This will give you an easy reference and benchmark baseline.
Click to expand...

Adrian, I assume your GitHub repo here:

https://github.com/CNMAT/OSC

…is the best place for me to start if I’m looking for a full-featured Teensy 4.1 - compatible OSC solution?

Have there been previous attempts by yourself or others to add audio?
 
celoranta said:
Adrian, I assume your GitHub repo here:

https://github.com/CNMAT/OSC

…is the best place for me to start if I’m looking for a full-featured Teensy 4.1 - compatible OSC solution?

Have there been previous attempts by yourself or others to add audio?
Click to expand...

Not very much that I have seen. I have just set up the hardware to do some OSC-controlled synths with Teensy 4.1 at my ponderous weekend-project rate.
 
Status
Not open for further replies.
Back
Top