Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 23 of 23

Thread: Any work with USB audio?

  1. #1
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12

    Any work with USB audio?

    Hi,
    I am brand new to Teensy; although I've been wanting to play with it for years.
    Now with the 4.0 version and high-speed USB ports, I'm hoping to build up a multi channel
    USB audio device that would plug into a Linux box and show up as an ALSA device.
    I browsed the forums and didn't see anything on this; so I apologize if I missed it.

    The ideal project (for me) would be to have the CS42448 board
    plug into the Teensy4.0 and look like a 6-in/8-out USB audio device when plugged into
    a Linux box. I've done a lot of ALSA work on Linux but I've not done any USB work on
    the device side, so without really digging in I don't even know if this is possible.

    Any thoughts or pointers?

  2. #2
    The USB audio profile has not been enabled in the T4 codebase yet. Taking a quick glance at the T3 codebase, I only see stereo support, no support for multichannel.

  3. #3
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    9,697
    Search forum for "CS42448" - there were notes about it being used in some fashion that may be a starting point.

    But indeed only USB Serial is working on T4 at this time

  4. #4
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    Ok, @wcalvert, but that's still good news to me...
    That implies that there is some support for audio streaming over USB...
    Maybe I should start with a pre-T4 board, to see what already works and
    then move up from there...
    Is the USB stuff from an open-source stack or written from scratch?
    Tx

  5. #5
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    @defragster...
    Yea, I saw some of these posts but as far as I could tell, they were not for streaming USB audio.
    But, again, the good news is if USB Serial is working without a FTDI-ish chip in front, then there
    is a firmware based USB stack in place...
    Tx

  6. #6
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,565
    Quote Originally Posted by edsut View Post
    Is the USB stuff from an open-source stack or written from scratch?
    It's all written from scratch, pretty much by me.

    Just this morning I wrote this message about the state of non-Serial USB stuff on Teensy 4.0.

    I definitely planning to implement 2 audio options on Teensy 4.0... one stereo that's pretty much the same as we have on Teensy 3.x, and a "many channels" option. Many decisions remain on this stuff, especially whether to stick with asynchronous rate feedback for the isochronous streaming, or to switch to adaptive mode, since we have a much more powerful CPU to resample the data.

    I can tell you I will do pretty much all the other USB types before USB Audio.

    If you're planning to ultimately use Linux, you might also consider whether you'll use PulseAudio on the Linux machine. Perhaps the things will be easier with Teensy 4.0 since we have a PLL dedicated to audio so we can get a very precise 44100 Hz sample rate. But on the other boards where we're at 44117 Hz, Linux systems without PulseAudio (like Raspberry Pi) tend to perform very badly due to the sample rate mismatch.

  7. #7
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    @PaulStoffregen
    Sheesh, seems like you've got plenty on your plate (understatement)!
    Is all this stuff open source that I could contribute to (assuming I am able to do this project)?

    Yea, Linux is the backend of all this. I assume the clock issue on the 3.x boards is because it is derived from the system clock to avoid added expense/space for a dedicated external clock. True?

    Tx

  8. #8
    The code is open source, yes: https://github.com/PaulStoffregen/cores

    For the existing USB audio functionality, look for usb_audio.h/cpp under the teensy3 directory.

  9. #9
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    Thanks to all for the quick responses.

    Ok, so it seems safe to say that the USB side of things would be able to handle the 8x8 (ish)
    audio channel count (16bit words, 44.1kHz likely).

    I shifted gears and started looking at the audio (SAI) interfaces on the two
    CPUs (Teensy3.6/K66 vs Teensy4.0/RT1060) and now I'm nervous again...
    It appears that while the RT1060 blows away the K66 in performance, it has less
    SAI channel count capacity (hopefully I'm just not reading something correctly)...

    I raised the point in a different thread...
    and I posted a question to NXP's support page.
    If anyone already has experience with this, I'd sure be interested...
    Tx

  10. #10
    Senior Member
    Join Date
    Jul 2014
    Posts
    2,331
    @edsut
    SAI is somewhat different on T4 compared to T3
    first SAI has 1 dedicated input and output port and 3 shared IO data ports
    second SAI has 1 IN and 1 OUT data port
    So on SAI1 you can have up to 4 IN and 1 OUT; or 3 IN and 2 OUT; or 2 IN and 3 OUT; or 1 IN and 4 OUT;
    SAI2 has only 1 IN and 1 OUT data port

  11. #11
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,565
    and I posted a question to NXP's support page.
    Maybe give us a link to that page, so we can see how they answer?

  12. #12
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,565
    Please try not to spread your question over multiple threads on this forum. I know it must seem tempting, that by doing so you might reach a larger audience. But we're a fairly small community here where many of us use the "What's new" page or other ways of looking at everything. Duplicates only annoy people who would be the best at helping.

    From that other thread, you wrote (in detail you could have said here):

    For Teensy 3.6, referring to https://www.pjrc.com/teensy/K66P144M180SF5RMV2.pdf, section 61.2.1...
    ... Each SAI data line may support 1-32 audio channels...
    For Teensy 4.0, referring to https://www.pjrc.com/teensy/IMXRT1060RM_rev1.pdf, section 36.1.1 (top of pg 2067)...
    ... SAI-1 is used for multi-channel audio interface, which supports up to 8-channels audio in or out...
    Both of these statements are a bit misleading, especially the first.

    The first statement is talking about the possibility to use different protocols. Of the well defined protocols (the ones where you can actually buy ADC, DAC, Codec chips), all of them are 2 audio channel per pin, except TDM. But to say TDM can be used for 32 channels is quite a stretch. On Teensy 3.x the maximum BCLK is 12.5 MHz, which means at most a 256 bit TDM frame for ordinary audio sample rates. If you divvy up 256 bits until 32 samples, that's only 8 bit resolution.

    I2S and several I2S-like protocols (with names like "left justified") are always 2 channels. These protocols are by far the most commonly used. Use of TDM is possible, but relatively rare. Talk of digital audio between chips usually revolves about an assumption of I2S protocol, like the 2nd statement. But even this statement revolves around the word "or" since SAI1 has 5 data pins, not 8.

    How many simultaneous TDM streams you can really use on Teensy 4.0 is still an open question. Whether 512 bit frame size can work is also uncertain (but looks promising, other than few chips would support it). Maybe Frank will see this and have an opinion? I'm very familiar with the hardware and I'm still not certain. You really can't always tell from these statements in the summary of features. The only way to really know is to dive into actually writing and testing the code.

    So with that in mind, I would like to ask... do you realistically see yourself getting into writing new driver code for the SAI hardware? It's quite advanced work. Very few people have managed to do it. If you do, will you be sharing as open source, maybe even contribute back to the audio library?

  13. #13
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    @WMXZ...
    I am playing catchup here, not sure what "port" vs "channel" means so
    at the moment I'm too dumb to comment. Bottom line is my project has
    a few different use cases, three of them will probably work just fine
    1: 8-audio channels in (i.e. mics)
    2: 8-audio channels out (i.e. spkrs)
    3: 4-in/2-out (i.e. mics/spkrs)

    the fourth one is a leap to 16 or more input channels; but I may be able
    to just push that off (or even drop) for now.
    Tx

  14. #14
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    @PaulStoffregen
    First, please realize I am 3-4 days into thinking through this project,
    so if my "hopes and dreams" are absurd I claim full responsibility. :-)
    I jumped on this when I heard about the Teensy4.0 board.

    Regarding the NXP support page...
    I think you can "Invite" listeners to a case, I'll add you if I can
    (I may need your email address for that). If I can't add you in,
    (or if you prefer not to give me your email addr) I will certainly
    share the answer here.

    Regarding multiple threads...
    When I saw the CS42448 thread, that seemed more appropriate for the
    audio part of this discussion; and yes, I did put it there thinking it would reach
    a wider audience. My bad, feel free to move or delete my comment on that thread
    if you prefer.

    Regarding my work on this...
    I've done quite a bit of firmware at cpu bootloader and device interface level over
    the years. Have not worked on driver level USB or I2S, hence my initial concern
    regarding USB. I assumed I2S would be like any other interface... most time spent
    trying to figure out the datasheets and how to hook up a scope probe to a surface
    mount device :-).

    The project is for some research my group is doing. The channel counts will vary
    and if I can eliminate one of the cases mentioned above, then this may
    just work as is. If I get approval, then I may end up building up the platform to run
    in one of three modes: 8-in OR 8-out OR 4-in/2-out (all likely to be 16bit/44.1kHz).
    There is one other scenario that I was hoping to cover that would involve many more
    input channels, but that just may be out of reach for this anyway.

    Regarding contributing back...
    I can say with reasonable certainty that any USB/SAI work I do that would be
    reusable to the community I would be able to contribute back. That's not the secret
    sauce.

    Thanks for the responses.

  15. #15
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    9,697
    Quote Originally Posted by edsut View Post
    @defragster...
    Yea, I saw some of these posts but as far as I could tell, they were not for streaming USB audio.
    But, again, the good news is if USB Serial is working without a FTDI-ish chip in front, then there
    is a firmware based USB stack in place...
    Tx
    USB Audio will come online - reference to those CS42448 posts was to see what was already working on T_3.6 as that ideally will be fully T4 supported. There is an OSH board among those references in some of the posts - IIRC posted by Paul@PJRC so that should be a good baseline and somebody already got that hardware interface to T4 ( again ) IIRC in those posts - getting the data out over USB will follow as possible.

  16. #16
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    I got a response from NXP.
    Here is the dialog:

    Question I posted:

    I am considering using the iMXRT1060 for a multi-channel audio-to-USB device.
    While the exact channel count is yet to be determined it hovers around 8-12 input
    channels and 8-12 output channels 16-bit PCM at 44kHz. This number is relatively
    small compared to the specs of most of the SAI interfaces on other cortex-M
    processors, so I kinda assumed the iMXRT1060 would have no problem supporting
    that; however, the reference manual (section 36.1.1, pg 2067) has the following statement:

    SAI-1 is used for multi-channel audio interface, which supports up to 8-channels
    audio input or 8-channels audio output at 384 kHz/32-bit. SAI-2 and SAI-3 can be
    used for stereo audio input and output up to 384 kHz/32-bit. Also SAI-3 is able to
    drive MQS directly as a low-cost audio output.

    That seems to imply that instead of the typical 32 I/O channels per SAI interface,
    this device has only 8, and only in OR out, not a combination. Can this be clarified
    somehow? If this really is a limitation, is there any other "crossover" Cortex M7
    that supports more channels?
    Answer I received:

    There is an application note AN12090 about RT1060 Using Multi-Channel Feature of SAI.
    It shows how to use SAI1 4TX pins to support 7.1 surround sound (8 channels) use case.
    Please check more detailed info from below link:
    https://www.nxp.com/docs/en/applicat...te/AN12090.pdf

    I could not find any i.MXRT product has one SAI module support 32 channels audio input/output.
    Customer could consider to use i.MXRT FlexIO module, which supports I2S bus.
    Customer could extend the transfer pin number and receive pin number as required.
    More detailed info about FlexIO module, please check RT1060 reference manual chapter49.
    I have not had a chance to look at their pointers; maybe over the weekend...

  17. #17
    Senior Member
    Join Date
    Jul 2014
    Posts
    2,331
    Customer could consider to use i.MXRT FlexIO module, which supports I2S bus.
    I have to look into that!
    Last edited by WMXZ; 09-20-2019 at 03:02 PM. Reason: Removed edit

  18. #18
    Senior Member
    Join Date
    Oct 2012
    Location
    Portland OR
    Posts
    676
    FWIW, I notice the iMXRT1060 p.2067 claim you quote is 8 channels at 384 kHz / 32 bit. By my calculation that is 12.29 Mbit/sec per channel, or 98.3 Mbit/sec for 8 channels, if you can really do all 8 at the same time.

    By contrast, 12 channels of 44.1 kHz / 16 bits is 705.6 kbit/s per channel, or a total of 8.47 Mbit/sec for all 12 channels. So strictly by bitrate it might appear that 12 channels of "CD-quality" audio is much less demanding. I guess there is something about the channel multiplexing that has other limitations.

  19. #19
    Quote Originally Posted by PaulStoffregen View Post
    Linux systems without PulseAudio (like Raspberry Pi) tend to perform very badly due to the sample rate mismatch.
    Not that I use it, but I'm pretty sure you'll find PulseAudio does work ok on a Pi nowadays.

  20. #20
    Quote Originally Posted by JBeale View Post
    FWIW, I notice the iMXRT1060 p.2067 claim you quote is 8 channels at 384 kHz / 32 bit. By my calculation that is 12.29 Mbit/sec per channel, or 98.3 Mbit/sec for 8 channels, if you can really do all 8 at the same time.

    By contrast, 12 channels of 44.1 kHz / 16 bits is 705.6 kbit/s per channel, or a total of 8.47 Mbit/sec for all 12 channels. So strictly by bitrate it might appear that 12 channels of "CD-quality" audio is much less demanding. I guess there is something about the channel multiplexing that has other limitations.
    In the first case, the data setup time of 15nS makes this unlikely to work as you are thinking. In fact SAI1 outputs 8 channels as 4 streams of 2 channels over the 4 active outputs (or active inputs if you prefer) so the clocking is far less onerous. And the next line seems to imply that SAI2 and 3 only like stereo channels as well.

    If you want to do large number of SAI channels on single links you're probably better off with the latest STMicroelectronics H series chips, but they have lots of other problems.
    Last edited by MikeDB; 09-21-2019 at 12:14 AM.

  21. #21
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    @JBeale
    Yea, I don't know if the 8/384/32 is a bandwidth limit or channel count limit. If it is just implying a bandwidth max, then certainly a lot more channels will run when at 44.1kHz/16.

  22. #22
    Junior Member
    Join Date
    Sep 2019
    Location
    United States
    Posts
    12
    @MikeDB
    I would be quite happy just having 8in and 8out (simultaneously) at 44.1/16.
    The text I quoted uses 'or' so it kinda implies one or the other...

  23. #23
    Quote Originally Posted by edsut View Post
    @MikeDB
    I would be quite happy just having 8in and 8out (simultaneously) at 44.1/16.
    The text I quoted uses 'or' so it kinda implies one or the other...
    I'm afraid I read it as SAI1 offers 8 in, 2out through to 2 in, 8 out, i.e. 10 channels in total, not 16. But I think you can have these at 384kHz if you want.
    But you can use the SAI2 for another 2 in and 2 out and the SPDIF for yet another 2 in and 2 out, so you can sort of get there in a messy way

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •