Opus codec library

Status
Not open for further replies.

Zen_spartan

New member
Forgive me for my newbieness but is the reason for there not being an opus or speex library available for Teensyduino? Am I barking up the wrong tree trying to use a teensy 3.2 for this purpose? I'm familiar developing for Linux based systems, though this is my first micro controller project.
If it's the case that simply no one attempted it yet, if anyone has a good starting point I don't mind giving it a try.

The project:
I'm looking to encode audio at around 16kbs for transmission across a wireless link for post processing.
 
Short answer is that nobody has needed one. Would certainly be possible but at some point with audio processing, especially if the end point is ethernet it's quicker to use a Linux SBC and an off the shelf library to get it done and I think that's where most people ended up, even though the up front hardware cost is larger.

Looking at a similar project I tried the queue object in the audio library

https://www.pjrc.com/teensy/gui/?info=AudioRecordQueue
https://forum.pjrc.com/threads/45788-Working-Audio-output-queue-example

Which does allow you to get audio data into/out your code without writing a custom object. Do be aware that to run on a micro controller the audio library hard codes a number of things like sample rate and size which may not fit your needs for onforwarding, hence the trend to use SBC for this sort of thing despite the other problems they bring.
 
Interesting, I suppose with something like the pi zero the path of least resistance is too tempting! But where's the fun in that ! I'm going to have a go to see if I can make a library anyway, there seems to be library tutorials using the arduino ide. Hopefully I'll learn something along the way.
 
For what it's worth, the library part isn't too much of a hassle, the meaty part will be getting your audio data into the code and processing it within the constraints of a micro controller.
 
Opus is on my list, since a long time (mentioned it in the readme of my other codecs) ... if only the day had more hours...
On the other hand, till now, nobody requested it.
 
Last edited:
I've used opus on many sbc projects before and I'm always suprised when I mention it to folks how it's flown under a lot of people's radar especially with the results it achieves.
 
Just wondering whether I should begin this work or is there enough interest for one of the far more experienced guys to revise their excisting code?
 
I got the opus decoder to build in the Arduino IDE with a lot of include path hacks and file moves. What I can say is this:

First, it's hard to build. To decode opus, you need libogg, libopus and libopusfile. All three of them use autoconf but without the ability to add compiler flags to define macros I had to edit every source file that includes the auto-generated config.h (which BTW has to be generated by running ./configure on my computer and copied to the Arduino project). Also, libopus has headers in various directories, and without the ability to add include paths, I had to throw all the files into the same directory. I will wait until the Arduino IDE gets the ability to add extra include paths before trying to redo it properly.

Second, it uses quite a bit of RAM. I tested using op_open_memory which opens an opus file stored in an array. The lowest RAM usage I've seen from the op_open_memory call is 84k, but I've also seen it go higher than 128k and I don't know why. You need at least a teensy 3.5/3.6 to use it.

Speed wise, it's fine. Without any assembly optimizations (I deleted all the ASM so I don't have to deal with it), a teensy 3.6 at 180MHz can decode opus at about 5x speed.
 
Cool.. i ever wanted that, but never found the time.
Do you have the encoder, too? Is the sampling-rate conversion working?
 
I deleted all the files that look like they're for the encoder to get something building sooner. The opus decoder doesn't have a sample rate converter. The opusfile docs recommend the speex resampler. I plan to just change the I2S registers to make the output run at 48kHz.
 
I looked into the memory usage more. I think a lot of it comes from the ogg demuxer. If I call the underlying opus_decoder_create, it only allocates 28k of memory.

The problem is this: The ogg format consists of pages, which can be up to 64kB. Each frame can contain multiple packets (a packet corresponds to a frame of compressed audio) and each packet can also span multiple pages. The ogg demuxer has 2 layers - the frame sync layer and the stream layer. The frame sync layer reads data from the file and outputs pages, and the stream layer takes pages from the frame sync layer and outputs packets. The frame sync layer has a page buffer, and the stream layer makes a copy of the pages it reads in so it also has its own buffer. This means we need to reserve 128kB just to read ogg files! There is some discussion about this in this rant and this response to the rant.

There is a zero-copy libogg2 that doesn't make a copy in the stream layer, so it would only need 64kB, but that xiph seem to have abandoned libogg2. It only exists in their old https://svn.xiph.org/ and not the new https://git.xiph.org/.

There's also the option of writing our own ogg demuxer that only reads the page header into memory instead of the whole page. I'm using this code to check the page and packet sizes. The header size seems to stay below 512 bytes, and individual vorbis or opus packets at max bitrate don't seem to go over 4kB (opus doesn't even go above 1500 bytes).
 
I got the low-mem ogg reader done and cleaned up the libopus build. I also enabled the assembly optimizations but it didn't make a noticeable difference.

The code size is about 100k and the memory usage is about 50k. A teensy 3.5/3.6/4 will be required because of the memory usage. Low bitrate files may play fine on 48MHz (I tested 128k) but 72MHz is needed to play high bitrate files.

Get the code here: https://github.com/jcj83429/Arduino-Teensy-Codec-lib. There's an example in the examples folder.

Supported:
- mono and stereo
- frame sizes from 2.5ms to 20ms (20ms is the default and the most common)
- all bitrates (8 to 512kbps)

Not supported
- multichannel files
- frame sizes larger than 20ms (can be supported by increasing buffer sizes but I think it's not worth supporting. The high quality CELT layer doesn't benefit from >20ms frame sizes)
- resampling to 44100Hz. The I2S output rate needs to be changed to 48000. My example code does this. IDK how much CPU will be needed to resample.

Not yet supported
- ogg/opus duration parsing
- ogg/opus seeking (I added seeking for mp3/aac/flac)
- ogg/opus replaygain (I added replaygain for mp3/aac/flac)
- yielding to user code more frequently
 
I am looking into the possibility to make a mumble voice chat client on the Teensy 4.1. My reasoning not to into the RPI route is the excellent Teensy hardware (low-power), available in a compact form-factor, especially with the native ethernet connection. I'll probably start a separate thread to probe interest and explain other specs (POE amongst others).

Is anybody aware of a more advanced implementation of the OPUS codec than one listed above?
For a mumble implementation, i need both encoder and decoder to be available, but i do not need any ogg file reading / manipulation.
 
If you don't need to deal with files then it's actually simpler. Most of the code I wrote for opus has to do with ogg files. You just need to call the libopus encode/decode function to turn PCM samples into opus frames and back.
 
Hi all, coding is not really my thing, but lockdown has a way of encouraging new skills.
I found an nRF SDK with opus-1.2.1 and managed to compile to run on the nrf52840 dongle.
Somewhat dissatisfied with my success (I couldn't listen to it!), it made sense to try it on the Teensy 4.0 (what an awesome platform).
Just followed PJRC's recommendations for creating objects and ta-da;

https://github.com/mgergos/Opus4Teensy

Enjoy...
 
Status
Not open for further replies.
Back
Top