Any work with USB audio?

Status
Not open for further replies.

edsut

Well-known member
Hi,
I am brand new to Teensy; although I've been wanting to play with it for years.
Now with the 4.0 version and high-speed USB ports, I'm hoping to build up a multi channel
USB audio device that would plug into a Linux box and show up as an ALSA device.
I browsed the forums and didn't see anything on this; so I apologize if I missed it.

The ideal project (for me) would be to have the CS42448 board
plug into the Teensy4.0 and look like a 6-in/8-out USB audio device when plugged into
a Linux box. I've done a lot of ALSA work on Linux but I've not done any USB work on
the device side, so without really digging in I don't even know if this is possible.

Any thoughts or pointers?
 
The USB audio profile has not been enabled in the T4 codebase yet. Taking a quick glance at the T3 codebase, I only see stereo support, no support for multichannel.
 
Search forum for "CS42448" - there were notes about it being used in some fashion that may be a starting point.

But indeed only USB Serial is working on T4 at this time
 
Ok, @wcalvert, but that's still good news to me...
That implies that there is some support for audio streaming over USB...
Maybe I should start with a pre-T4 board, to see what already works and
then move up from there...
Is the USB stuff from an open-source stack or written from scratch?
Tx
 
@defragster...
Yea, I saw some of these posts but as far as I could tell, they were not for streaming USB audio.
But, again, the good news is if USB Serial is working without a FTDI-ish chip in front, then there
is a firmware based USB stack in place...
Tx
 
Is the USB stuff from an open-source stack or written from scratch?

It's all written from scratch, pretty much by me.

Just this morning I wrote this message about the state of non-Serial USB stuff on Teensy 4.0.

I definitely planning to implement 2 audio options on Teensy 4.0... one stereo that's pretty much the same as we have on Teensy 3.x, and a "many channels" option. Many decisions remain on this stuff, especially whether to stick with asynchronous rate feedback for the isochronous streaming, or to switch to adaptive mode, since we have a much more powerful CPU to resample the data.

I can tell you I will do pretty much all the other USB types before USB Audio.

If you're planning to ultimately use Linux, you might also consider whether you'll use PulseAudio on the Linux machine. Perhaps the things will be easier with Teensy 4.0 since we have a PLL dedicated to audio so we can get a very precise 44100 Hz sample rate. But on the other boards where we're at 44117 Hz, Linux systems without PulseAudio (like Raspberry Pi) tend to perform very badly due to the sample rate mismatch.
 
@PaulStoffregen
Sheesh, seems like you've got plenty on your plate (understatement)!
Is all this stuff open source that I could contribute to (assuming I am able to do this project)?

Yea, Linux is the backend of all this. I assume the clock issue on the 3.x boards is because it is derived from the system clock to avoid added expense/space for a dedicated external clock. True?

Tx
 
Thanks to all for the quick responses.

Ok, so it seems safe to say that the USB side of things would be able to handle the 8x8 (ish)
audio channel count (16bit words, 44.1kHz likely).

I shifted gears and started looking at the audio (SAI) interfaces on the two
CPUs (Teensy3.6/K66 vs Teensy4.0/RT1060) and now I'm nervous again...
It appears that while the RT1060 blows away the K66 in performance, it has less
SAI channel count capacity (hopefully I'm just not reading something correctly)...

I raised the point in a different thread...
and I posted a question to NXP's support page.
If anyone already has experience with this, I'd sure be interested...
Tx
 
@edsut
SAI is somewhat different on T4 compared to T3
first SAI has 1 dedicated input and output port and 3 shared IO data ports
second SAI has 1 IN and 1 OUT data port
So on SAI1 you can have up to 4 IN and 1 OUT; or 3 IN and 2 OUT; or 2 IN and 3 OUT; or 1 IN and 4 OUT;
SAI2 has only 1 IN and 1 OUT data port
 
Please try not to spread your question over multiple threads on this forum. I know it must seem tempting, that by doing so you might reach a larger audience. But we're a fairly small community here where many of us use the "What's new" page or other ways of looking at everything. Duplicates only annoy people who would be the best at helping.

From that other thread, you wrote (in detail you could have said here):

For Teensy 3.6, referring to https://www.pjrc.com/teensy/K66P144M180SF5RMV2.pdf, section 61.2.1...
... Each SAI data line may support 1-32 audio channels...
For Teensy 4.0, referring to https://www.pjrc.com/teensy/IMXRT1060RM_rev1.pdf, section 36.1.1 (top of pg 2067)...
... SAI-1 is used for multi-channel audio interface, which supports up to 8-channels audio in or out...

Both of these statements are a bit misleading, especially the first.

The first statement is talking about the possibility to use different protocols. Of the well defined protocols (the ones where you can actually buy ADC, DAC, Codec chips), all of them are 2 audio channel per pin, except TDM. But to say TDM can be used for 32 channels is quite a stretch. On Teensy 3.x the maximum BCLK is 12.5 MHz, which means at most a 256 bit TDM frame for ordinary audio sample rates. If you divvy up 256 bits until 32 samples, that's only 8 bit resolution.

I2S and several I2S-like protocols (with names like "left justified") are always 2 channels. These protocols are by far the most commonly used. Use of TDM is possible, but relatively rare. Talk of digital audio between chips usually revolves about an assumption of I2S protocol, like the 2nd statement. But even this statement revolves around the word "or" since SAI1 has 5 data pins, not 8.

How many simultaneous TDM streams you can really use on Teensy 4.0 is still an open question. Whether 512 bit frame size can work is also uncertain (but looks promising, other than few chips would support it). Maybe Frank will see this and have an opinion? I'm very familiar with the hardware and I'm still not certain. You really can't always tell from these statements in the summary of features. The only way to really know is to dive into actually writing and testing the code.

So with that in mind, I would like to ask... do you realistically see yourself getting into writing new driver code for the SAI hardware? It's quite advanced work. Very few people have managed to do it. If you do, will you be sharing as open source, maybe even contribute back to the audio library?
 
@WMXZ...
I am playing catchup here, not sure what "port" vs "channel" means so
at the moment I'm too dumb to comment. Bottom line is my project has
a few different use cases, three of them will probably work just fine
1: 8-audio channels in (i.e. mics)
2: 8-audio channels out (i.e. spkrs)
3: 4-in/2-out (i.e. mics/spkrs)​

the fourth one is a leap to 16 or more input channels; but I may be able
to just push that off (or even drop) for now.
Tx
 
@PaulStoffregen
First, please realize I am 3-4 days into thinking through this project,
so if my "hopes and dreams" are absurd I claim full responsibility. :)
I jumped on this when I heard about the Teensy4.0 board.

Regarding the NXP support page...
I think you can "Invite" listeners to a case, I'll add you if I can
(I may need your email address for that). If I can't add you in,
(or if you prefer not to give me your email addr) I will certainly
share the answer here.​

Regarding multiple threads...
When I saw the CS42448 thread, that seemed more appropriate for the
audio part of this discussion; and yes, I did put it there thinking it would reach
a wider audience. My bad, feel free to move or delete my comment on that thread
if you prefer.​

Regarding my work on this...
I've done quite a bit of firmware at cpu bootloader and device interface level over
the years. Have not worked on driver level USB or I2S, hence my initial concern
regarding USB. I assumed I2S would be like any other interface... most time spent
trying to figure out the datasheets and how to hook up a scope probe to a surface
mount device :).

The project is for some research my group is doing. The channel counts will vary
and if I can eliminate one of the cases mentioned above, then this may
just work as is. If I get approval, then I may end up building up the platform to run
in one of three modes: 8-in OR 8-out OR 4-in/2-out (all likely to be 16bit/44.1kHz).
There is one other scenario that I was hoping to cover that would involve many more
input channels, but that just may be out of reach for this anyway.​

Regarding contributing back...
I can say with reasonable certainty that any USB/SAI work I do that would be
reusable to the community I would be able to contribute back. That's not the secret
sauce.​

Thanks for the responses.
 
@defragster...
Yea, I saw some of these posts but as far as I could tell, they were not for streaming USB audio.
But, again, the good news is if USB Serial is working without a FTDI-ish chip in front, then there
is a firmware based USB stack in place...
Tx

USB Audio will come online - reference to those CS42448 posts was to see what was already working on T_3.6 as that ideally will be fully T4 supported. There is an OSH board among those references in some of the posts - IIRC posted by Paul@PJRC so that should be a good baseline and somebody already got that hardware interface to T4 ( again ) IIRC in those posts - getting the data out over USB will follow as possible.
 
I got a response from NXP.
Here is the dialog:

Question I posted:

I am considering using the iMXRT1060 for a multi-channel audio-to-USB device.
While the exact channel count is yet to be determined it hovers around 8-12 input
channels and 8-12 output channels 16-bit PCM at 44kHz. This number is relatively
small compared to the specs of most of the SAI interfaces on other cortex-M
processors, so I kinda assumed the iMXRT1060 would have no problem supporting
that; however, the reference manual (section 36.1.1, pg 2067) has the following statement:

SAI-1 is used for multi-channel audio interface, which supports up to 8-channels
audio input or 8-channels audio output at 384 kHz/32-bit. SAI-2 and SAI-3 can be
used for stereo audio input and output up to 384 kHz/32-bit. Also SAI-3 is able to
drive MQS directly as a low-cost audio output.

That seems to imply that instead of the typical 32 I/O channels per SAI interface,
this device has only 8, and only in OR out, not a combination. Can this be clarified
somehow? If this really is a limitation, is there any other "crossover" Cortex M7
that supports more channels?

Answer I received:

There is an application note AN12090 about RT1060 Using Multi-Channel Feature of SAI.
It shows how to use SAI1 4TX pins to support 7.1 surround sound (8 channels) use case.
Please check more detailed info from below link:
https://www.nxp.com/docs/en/application-note/AN12090.pdf

I could not find any i.MXRT product has one SAI module support 32 channels audio input/output.
Customer could consider to use i.MXRT FlexIO module, which supports I2S bus.
Customer could extend the transfer pin number and receive pin number as required.
More detailed info about FlexIO module, please check RT1060 reference manual chapter49.

I have not had a chance to look at their pointers; maybe over the weekend...
 
FWIW, I notice the iMXRT1060 p.2067 claim you quote is 8 channels at 384 kHz / 32 bit. By my calculation that is 12.29 Mbit/sec per channel, or 98.3 Mbit/sec for 8 channels, if you can really do all 8 at the same time.

By contrast, 12 channels of 44.1 kHz / 16 bits is 705.6 kbit/s per channel, or a total of 8.47 Mbit/sec for all 12 channels. So strictly by bitrate it might appear that 12 channels of "CD-quality" audio is much less demanding. I guess there is something about the channel multiplexing that has other limitations.
 
FWIW, I notice the iMXRT1060 p.2067 claim you quote is 8 channels at 384 kHz / 32 bit. By my calculation that is 12.29 Mbit/sec per channel, or 98.3 Mbit/sec for 8 channels, if you can really do all 8 at the same time.

By contrast, 12 channels of 44.1 kHz / 16 bits is 705.6 kbit/s per channel, or a total of 8.47 Mbit/sec for all 12 channels. So strictly by bitrate it might appear that 12 channels of "CD-quality" audio is much less demanding. I guess there is something about the channel multiplexing that has other limitations.

In the first case, the data setup time of 15nS makes this unlikely to work as you are thinking. In fact SAI1 outputs 8 channels as 4 streams of 2 channels over the 4 active outputs (or active inputs if you prefer) so the clocking is far less onerous. And the next line seems to imply that SAI2 and 3 only like stereo channels as well.

If you want to do large number of SAI channels on single links you're probably better off with the latest STMicroelectronics H series chips, but they have lots of other problems.
 
Last edited:
@JBeale
Yea, I don't know if the 8/384/32 is a bandwidth limit or channel count limit. If it is just implying a bandwidth max, then certainly a lot more channels will run when at 44.1kHz/16.
 
@MikeDB
I would be quite happy just having 8in and 8out (simultaneously) at 44.1/16.
The text I quoted uses 'or' so it kinda implies one or the other...
 
@MikeDB
I would be quite happy just having 8in and 8out (simultaneously) at 44.1/16.
The text I quoted uses 'or' so it kinda implies one or the other...

I'm afraid I read it as SAI1 offers 8 in, 2out through to 2 in, 8 out, i.e. 10 channels in total, not 16. But I think you can have these at 384kHz if you want.
But you can use the SAI2 for another 2 in and 2 out and the SPDIF for yet another 2 in and 2 out, so you can sort of get there in a messy way
 
Status
Not open for further replies.
Back
Top