Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 15 of 15

Thread: Opus codec library

  1. #1
    Junior Member
    Join Date
    Sep 2017
    Posts
    4

    Opus codec library

    Forgive me for my newbieness but is the reason for there not being an opus or speex library available for Teensyduino? Am I barking up the wrong tree trying to use a teensy 3.2 for this purpose? I'm familiar developing for Linux based systems, though this is my first micro controller project.
    If it's the case that simply no one attempted it yet, if anyone has a good starting point I don't mind giving it a try.

    The project:
    I'm looking to encode audio at around 16kbs for transmission across a wireless link for post processing.

  2. #2
    Senior Member
    Join Date
    Apr 2013
    Posts
    1,927
    Short answer is that nobody has needed one. Would certainly be possible but at some point with audio processing, especially if the end point is ethernet it's quicker to use a Linux SBC and an off the shelf library to get it done and I think that's where most people ended up, even though the up front hardware cost is larger.

    Looking at a similar project I tried the queue object in the audio library

    https://www.pjrc.com/teensy/gui/?info=AudioRecordQueue
    https://forum.pjrc.com/threads/45788...-queue-example

    Which does allow you to get audio data into/out your code without writing a custom object. Do be aware that to run on a micro controller the audio library hard codes a number of things like sample rate and size which may not fit your needs for onforwarding, hence the trend to use SBC for this sort of thing despite the other problems they bring.

  3. #3
    Junior Member
    Join Date
    Sep 2017
    Posts
    4
    Interesting, I suppose with something like the pi zero the path of least resistance is too tempting! But where's the fun in that ! I'm going to have a go to see if I can make a library anyway, there seems to be library tutorials using the arduino ide. Hopefully I'll learn something along the way.

  4. #4
    Senior Member
    Join Date
    Apr 2013
    Posts
    1,927
    For what it's worth, the library part isn't too much of a hassle, the meaty part will be getting your audio data into the code and processing it within the constraints of a micro controller.

  5. #5
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    21,829
    Speex has been on my long list of "look at it someday" audio stuff. I believe you're the first person to actually mention it.

    I've never heard of the opus codec before.

    If you're going to try adding to the audio library, which can be done as a separate lib too, start here:

    https://www.pjrc.com/teensy/td_libs_...ewObjects.html

  6. #6
    Senior Member
    Join Date
    Apr 2013
    Posts
    1,927
    @ paul, looking at the header here, Opus is supposed to be a successor.
    https://www.speex.org/
    Bigger, more powerful and more flexible may not be what one wants when implementing into a micro of course.

  7. #7
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    6,886
    Opus is on my list, since a long time (mentioned it in the readme of my other codecs) ... if only the day had more hours...
    On the other hand, till now, nobody requested it.
    Last edited by Frank B; 09-02-2017 at 10:19 PM.

  8. #8
    Junior Member
    Join Date
    Sep 2017
    Posts
    4
    I've used opus on many sbc projects before and I'm always suprised when I mention it to folks how it's flown under a lot of people's radar especially with the results it achieves.

  9. #9
    Junior Member
    Join Date
    Sep 2017
    Posts
    4
    Just wondering whether I should begin this work or is there enough interest for one of the far more experienced guys to revise their excisting code?

  10. #10
    Junior Member
    Join Date
    May 2018
    Posts
    4
    Came across this thread and thought I'd express further interest.

  11. #11
    Junior Member
    Join Date
    Jan 2020
    Posts
    4
    I got the opus decoder to build in the Arduino IDE with a lot of include path hacks and file moves. What I can say is this:

    First, it's hard to build. To decode opus, you need libogg, libopus and libopusfile. All three of them use autoconf but without the ability to add compiler flags to define macros I had to edit every source file that includes the auto-generated config.h (which BTW has to be generated by running ./configure on my computer and copied to the Arduino project). Also, libopus has headers in various directories, and without the ability to add include paths, I had to throw all the files into the same directory. I will wait until the Arduino IDE gets the ability to add extra include paths before trying to redo it properly.

    Second, it uses quite a bit of RAM. I tested using op_open_memory which opens an opus file stored in an array. The lowest RAM usage I've seen from the op_open_memory call is 84k, but I've also seen it go higher than 128k and I don't know why. You need at least a teensy 3.5/3.6 to use it.

    Speed wise, it's fine. Without any assembly optimizations (I deleted all the ASM so I don't have to deal with it), a teensy 3.6 at 180MHz can decode opus at about 5x speed.

  12. #12
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    6,886
    Cool.. i ever wanted that, but never found the time.
    Do you have the encoder, too? Is the sampling-rate conversion working?

  13. #13
    Junior Member
    Join Date
    Jan 2020
    Posts
    4
    I deleted all the files that look like they're for the encoder to get something building sooner. The opus decoder doesn't have a sample rate converter. The opusfile docs recommend the speex resampler. I plan to just change the I2S registers to make the output run at 48kHz.

  14. #14
    Junior Member
    Join Date
    Jan 2020
    Posts
    4
    I looked into the memory usage more. I think a lot of it comes from the ogg demuxer. If I call the underlying opus_decoder_create, it only allocates 28k of memory.

    The problem is this: The ogg format consists of pages, which can be up to 64kB. Each frame can contain multiple packets (a packet corresponds to a frame of compressed audio) and each packet can also span multiple pages. The ogg demuxer has 2 layers - the frame sync layer and the stream layer. The frame sync layer reads data from the file and outputs pages, and the stream layer takes pages from the frame sync layer and outputs packets. The frame sync layer has a page buffer, and the stream layer makes a copy of the pages it reads in so it also has its own buffer. This means we need to reserve 128kB just to read ogg files! There is some discussion about this in this rant and this response to the rant.

    There is a zero-copy libogg2 that doesn't make a copy in the stream layer, so it would only need 64kB, but that xiph seem to have abandoned libogg2. It only exists in their old https://svn.xiph.org/ and not the new https://git.xiph.org/.

    There's also the option of writing our own ogg demuxer that only reads the page header into memory instead of the whole page. I'm using this code to check the page and packet sizes. The header size seems to stay below 512 bytes, and individual vorbis or opus packets at max bitrate don't seem to go over 4kB (opus doesn't even go above 1500 bytes).

  15. #15
    Junior Member
    Join Date
    Jan 2020
    Posts
    4
    I got the low-mem ogg reader done and cleaned up the libopus build. I also enabled the assembly optimizations but it didn't make a noticeable difference.

    The code size is about 100k and the memory usage is about 50k. A teensy 3.5/3.6/4 will be required because of the memory usage. Low bitrate files may play fine on 48MHz (I tested 128k) but 72MHz is needed to play high bitrate files.

    Get the code here: https://github.com/jcj83429/Arduino-Teensy-Codec-lib. There's an example in the examples folder.

    Supported:
    - mono and stereo
    - frame sizes from 2.5ms to 20ms (20ms is the default and the most common)
    - all bitrates (8 to 512kbps)

    Not supported
    - multichannel files
    - frame sizes larger than 20ms (can be supported by increasing buffer sizes but I think it's not worth supporting. The high quality CELT layer doesn't benefit from >20ms frame sizes)
    - resampling to 44100Hz. The I2S output rate needs to be changed to 48000. My example code does this. IDK how much CPU will be needed to resample.

    Not yet supported
    - ogg/opus duration parsing
    - ogg/opus seeking (I added seeking for mp3/aac/flac)
    - ogg/opus replaygain (I added replaygain for mp3/aac/flac)
    - yielding to user code more frequently

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •