48kHz 8i80 USB Audio

taita

Member
Hello,

I intend to use T4 build an 8i8o USB interface running at 48kHz/24bit. Hopefully this can undertake some of the work required to have 48kHz audio and 24/32bit support included in future TeensyDuino releases.

I'm new to Teensy but have done a fair bit of PCB design & building as well as a few successful XMOS usb audio devices, so I happily accept that a lot of refactoring the codebase is going to be part of this. PJRC has already done a great deal of excellent work on this already and so first up I acknowledge that as in all open-source projects we are standing on the shoulders of giants.

So after a dig through the TeensyDuino v1.52beta code my initial goals are :

1. Get the iMXRT1062 SAI1 to run at 48kHz, via PLL4, and provide MCLK + BCLK + LRCLK and interface TDM in & TDM out at the appropriate frequencies.

2. Ensure that the host enumerates the client properly (ie: recognises an 8i8o device running at 48kHz/24-bit).

3. Pass 24bit audio to/from the TDM data streams to/from the USB endpoint and host. Or, 24-bit padded to 32-bit depending on which is easier...initially.

4. Utilise CS42448 (6i8o) + the SAI3 spdif stereo input (2i) to achieve 8i8o.


My inital goals are NOT (but may be in future) :

1. Enabling volume, effects, multi-stage patching etc. My needs are very basic, and all such processing shall be done by the host.

2. Providing run-time user options to switch Fs or sample size. If you flash the device as a 48kHz/24bit device, that's how it will stay until you re-flash it.

3. Buffer size controls for "latency vs stability" optimisation. These would be useful in practise but latency is not a concern for me in this project.

4. To wax theoretical about the benefits of higher sampling rates and bit depths etc. I do not require a lesson on such matters. Audioholics forum is for that.

My ultimate goals are:

1. Contribute something useful to the TDAudio library

2. Achieve a nice stable USB 2.0 Class-compliant 8i8o USB interface that runs on Linux, because there are hardly any out there.

3. Maybe a few latency vs stability controls or the ability to switch between 44.1kHz and 48kHz at run-time


No doubt this list will change a bit but it's a starting point. Hopefully we can interest some contributors along the way.
 
There are no makefiles.

For 48kHz on Teensy 4, and just I2S you only need to edit this constant: https://github.com/PaulStoffregen/cores/blob/master/teensy4/AudioStream.h#L55 (not tested)
Some minor other things will not work (like the Guitar, USB-Audio or the other things mentioned above)
If you test it, please report back :)

Indeed AUDIO_SAMPLE_RATE_EXACT is referenced to setup the SAI clocking and PLL4 at : https://github.com/PaulStoffregen/Audio/blob/master/output_tdm.cpp#L284

That looks like the easy part! I have not yet recieved my T4 hardware (hopefully next week) so I would need to confirm on the scope that the MCLK + LRCLK + BCLK + DATAOUT lines are running at the right speed. If DATAOUT is okay I would presume DATAIN should be ready to recieve input under the same clock.

The next-easiest ( :confused:) part seems like getting the host/client enumeration right. My to-do for this is as follows (please correct me if I've totally missed or mistaken anything) :::

usb.c ::
- Ensure running at 480Mbit/sec

usb_desc.c ::
- Nominate correct input & output channel count and sampling frequency
- No preferred channel positions (like front, surround etc)
- Correct wMaxPacketSize for TX and RX (maybe after setting up 24/32 bit functions)
- Ensure descriptor sizes are corrected/changed if needed

usb_audio.h & usb_audio.cpp ::
- Increase channel count to 8 (maybe use arrays for channels rather than just multiple audio_block_t )
- Various other things like feedback_accumulator size etc

Seems like the really fiddly part will be revising all of the integer/block sizes all the way through the chain to accomodate the increased data size (like AudioStream / usb_desc / usb_audio ).
 
Looks like a solid plan and you seem to have prior experience to pull this off. Fingers crossed.
Are any of your XMOS interfaces open hardware by any chance? I'm looking into creating one myself and I could use an example project :)
 
- Increase channel count to 8 (maybe use arrays for channels rather than just multiple audio_block_t )
- Various other things like feedback_accumulator size etc

Seems like the really fiddly part will be revising all of the integer/block sizes all the way through the chain to accomodate the increased data size (like AudioStream / usb_desc / usb_audio ).

Please keep that up, so your work could be merged into Audio later without rewriting everything again.
This will also make it easy to do effects etc. Latency is anyway down to the bare minimum required by mathematics.
Problem is, Audio uses 16 bit samples for higher performace.
So it could be useful to define a new type of audio_block32_t for example and provide resampling blocks to interface with the rest of Audio, or everything will have to be rewritten, which also is only useful for T4, because of performace penalty excluding slower Teensies, so a new AudioT4, preferrably with switches for sample width and frequency, would be the best overall solution.
 
Looks like a solid plan and you seem to have prior experience to pull this off. Fingers crossed.
Are any of your XMOS interfaces open hardware by any chance? I'm looking into creating one myself and I could use an example project :)

No open hardware, but the XMOS evaluation boards are a good place to start
 
Okay so I have edited usb_desc.c and usb_desc_h to a stage where it is 'working' but goes only so far as to be successfully queried by the host (Win10 in this case) whereupon the Win10 host loads up the USB Audio Class 2.0 driver.

So far so good....however, it seems like the host then goes on to query the T4 in ways that it does not understand...I am guessing that the data structures defined in usb.c and possibly usb_audio.c do not successfully parse these Audio CLass 2.0 requests/responses and so enumeration ultimately fails, giving me a 'Teensy Audio' device in Windows Device Manager but with a yellow triangle. I am highly confident that my descriptors are okay because they match another USB 2.0 Audio card that I own, as read back packet-for-packet in Device Monitoring Studio (HHD Software). (Minus the clock selector and MIDI section).

Has anybody gotten so far as to reconfigure usb.c and/or usb_audio.co in order to correctly parse the requests and responses relevant to USB Audio Class 2.0? No problem if not, I'll just have to roll my sleeves up...
 
Okay so the (rather non-stack-like) stack didn't need a full re-write thankfully. The descriptors related to the '480MB/s' sections of usb_desc.h/c DID need to be completely rewritten though to conform with USB Audio Class 2.0 (along with the changes in descriptor size, etc). The UAC1 and UAC2 Interface Descriptors are very different from each-other.

Also needed to add a bunch of extra 'case' scenarios to the endpoint0_setup() in usb.c in order to properly parse certain UAC2 specific requests (like GetRangeSampleRate, GetCurrentSampleRate etc). Win10 and Linux can now both enumerate Teensy Audio as a UAC2 2i2o device running at 48kHz/24bit, and Wireshark shows isochronous data being thrown about when I use Win10 to send audio through it, and the Audio Streaming endpoints being switched back and forth between their Alternate settings during silence.

Code is still super-messy but will update here later. Next job will be to hook up the oscilloscope to probe what MCLK/LRCLK/BCLK/DATAO are doing....
 
Taita, where are you planning to get your CS42448 board from or do you already have one? I am trying to find something I can use with the T4 without SMD soldering. Ta!
 
You said that you wanted to be able to change the block size...

In my Float32 extension of the Audio library (for our "Tympan" project), I made the blocksize adjustable. While I did not make the USB code compatible with it (which is what you're interested in), I did make the I2S classes compatible with the adjustable block size.

Since I didn't touch the USB code, my actual code won't be very useful to you...but you might be interested in how I chose to expose this functionality to the Teensy programmer.

From this Tympan example: https://github.com/Tympan/Tympan_Li...1-Basic/ChangeSampleRate/ChangeSampleRate.ino
Code:
//set the sample rate and block size
const float sample_rate_Hz = 24000.0f ; //24000 or 44117 or 96000 (or other frequencies in the table in AudioOutputI2S_F32)
const int audio_block_samples = 32;     //do not make bigger than AUDIO_BLOCK_SAMPLES from AudioStream.h (which is 128)
AudioSettings_F32 audio_settings(sample_rate_Hz, audio_block_samples);

//create audio library objects for handling the audio
Tympan                    myTympan(TympanRev::D);   //do TympanRev::D or TympanRev::C
AudioInputI2S_F32         i2s_in(audio_settings);   //Digital audio *from* the Tympan AIC.
AudioEffectGain_F32       gain1, gain2;             //Applies digital gain to audio data.
AudioOutputI2S_F32        i2s_out(audio_settings);  //Digital audio *to* the Tympan AIC.  Always list last to minimize latency

So, you can see that I constructed a new class "AudioSettings_F32" that lets me set the sample rate and block size. This convenient object to pass to my other Audio classes (such as my I2S input and output classes) to inform them of these two critical parameters.

In the background, my new audio block class ("audio_block_f32_t") also knows its own block length and sample rate. So, in any given algorithm, you can always ask the audio block how long it is and what its sample rate is. By having these parameters attached to each audio block that is flowing through the system, it will allow me in the future to set up processing chains that might employ multiple different sample rates or block sizes.

Ideas for your consideration.

Chip
 
Hi,

Im on the exact same situation, wanting to port the USB 16bits implementation to 24/32 bits.

Is there anything in particular i could do to help on this task?
Else, im gonna start porting it, taking into account chipaudette's advice
 
The original poster has gone quiet I think but if you are able to take this on that would be amazing! Would be handy for many people on here
 
I would be really interested in quad channel audio output over USB, any progress people can make on this would also be amazing
 
I've been preoccupied with some other projects recently, looking at this one again. By way of update I did get the T4 to enumerate as a UAC2 device on Win10 and Debian but where I got stuck (and am revisiting now) was handling some standard USB2 requests that were not already baked into the UAC1 stack that comes with Teensyduino 1.52.

I did make some limited effort to identify the missing bits and 'patch' usb.c but as I patched I just kept finding that more and more bits were missing. So, I think I might just back away and re-write the UAC2 stack from scratch using a combination of the open source implementations out there, and try to port it to the IMXRT1062.
 
I've been preoccupied with some other projects recently, looking at this one again. By way of update I did get the T4 to enumerate as a UAC2 device on Win10 and Debian but where I got stuck (and am revisiting now) was handling some standard USB2 requests that were not already baked into the UAC1 stack that comes with Teensyduino 1.52.

I did make some limited effort to identify the missing bits and 'patch' usb.c but as I patched I just kept finding that more and more bits were missing. So, I think I might just back away and re-write the UAC2 stack from scratch using a combination of the open source implementations out there, and try to port it to the IMXRT1062.

I'm working on porting some code I have on a computer to a standalone device with audio input/audio output and some processing.
8 16/24/32 bit audio streams at either 48000khz or 96000khz would allow me to bypass the internal processing and have it offloaded to a dedicated computer or open up more possibilities as a generic dsp dac.
The current 2 channel usb input is workable but limits my use case to something very specific.

Is there some place I could view the code changes you have done so far?
I would be interested in discussing with you my plans, is there a place I would be better able to contact you?
 
I'm trying to understand the extent of the work required to implement this.

From what I gather, the following are required:
1. Update usb description to enumerate the correct number of channels
2. Modify the buffers to be the correct lengths
3. Modify the dma commands appropriately

What else do we think is required (Paul, your input here would be very much appreciated)?

The USB device already runs at high speed, right?
 
Yes, that's the general idea. I would add 2 more things to that list.

4: AudioStream (audio library) update() functions
5: Update the asynchronous rate feedback

But this simple list obscures some important details.

One decision to make right away is whether to have one big 704-720 byte packet per frame, or 8 smaller 80-90 byte packets (one on each microframe). Maybe you care about the impact you're having on the USB host's bandwidth scheduling algorithm and its ability to allow other periodic scheduled devices to work together smoothly on the same USB controller? Or maybe you just want to get it working the easiest way possible? If you decide to prefer good over easy, maybe you decide to handle 8000 interrupts per second? Or maybe you keep 1000 interrupts per second and schedule 8 packets by creating a chain of 8 qTD transfer descriptors (see page 2349 in the ref manual)?

Another detail to consider is what to do if 480 Mbit speed isn't available and the host completes enumeration at only 12 Mbit speed. The first challenge is Teensy's USB code assumes the full speed configuration descriptor is the same size and basically a mirror-like image of the high speed config descriptor. You'll probably need to extend the descriptor code to handle less-similar config descriptors for HS vs FS. Even if you don't care at all about full speed and know you will never use it, the USB protocol still requires both descriptors because the host can send a request to read the other speed's descriptor. See page 247 & 266 (and other parts of chapter 9) in the USB 2.0 spec for details.

When running at 12 Mbit, obviously you can't have two isochronous endpoints using 720 bytes per frame. I would imagine you'd use the same 180 byte size we have now and maybe add a little code to discard any buffers you get from the audio library on the other 6 channels. Not difficult, but do keep in mind as a high speed USB device you have no control over whether the host connects at 480 or 12 Mbit/sec. You must handle both cases.

As you can already see in the code, USB's asynchronous rate feedback is 3 bytes at full speed and 4 bytes at high speed. You'll probably modify only the high speed case. Just keep in mind 12 Mbit mode uses a 20 bit format versus a 32 bit format at 480 Mbit. Details on page 74 in the USB 2.0 spec.
 
Hi everyone! I am modifying the usb_desc.c to add up to four audio channels to input in the PC over USB. I am following the steps above, but I am a bit stuck. May anyone share the descriptor? Is it going to be incorporated multichannel usb audio in Audio System Design tool? Thanks in advance!
 
Hi everyone! I am modifying the usb_desc.c to add up to four audio channels to input in the PC over USB. I am following the steps above, but I am a bit stuck. May anyone share the descriptor? Is it going to be incorporated multichannel usb audio in Audio System Design tool? Thanks in advance!
I'm working on this right now too :) Though making it N channels for rx and M channels for tx. There's no reason to stop at 4. I'm refactoring the code to get rid of all references to 'left' and 'right', but making everything arrays of channels.

We'll see how it goes :)

If anybody's already got multi-channel USB working, I'd love to know.

Thanks,
-Caleb
 
Back
Top