Audio Library

Status
Not open for further replies.

PaulStoffregen

Well-known member
Edit: This URL is now the official Audio library page:

http://www.pjrc.com/teensy/td_libs_Audio.html

Much of the info on this thread is old, pre-1.0 stuff.

-----------------------------------------

Edit Again: This huge thread spans the pre-1.0 "beta" development of the Teensy Audio Library. Nearly all the issues mentioned were fixed before the 1.0 release. Most of the feature requests not implemented in 1.0 were put on the future development roadmap.

This old thread will be closed, now that the stable 1.0 release has been made.

-----------------------------------------

Here is the initial beta release of the audio library!

https://github.com/PaulStoffregen/Audio

Please understand this is *beta* status code. It works well, but many features are missing or incomplete, and it's very likely to get incompatible changes as features develop.

Currently the only "documentation" is the examples, and of course the source code itself.

If you discover any bugs, please try to post a reproducible bug report with complete code and enough detail to reproduce the problem.

The Teensy3 Audio Board, available and in stock as of Jan 3, 2014, is the primary hardware intended to be used with this library. It's also possible to use PWM output and ADC input.

teensy3_audio_front.jpg
teensy3_audio_back.jpg
 
Last edited:
Hi Paul

Thanks for all the hard work to make this available.

I cant find the "MemoryAndCpuUsage example" referred to in the examples code.

If the input is from the micro analog pin. it says...
// When AudioInputAnalog is running, analogRead() must NOT be used.

Also is it possible to stop the Audio library reading, to scan other ADC inputs, then restart the audio again?


Thanks
 
Ahh looks delicious.
I'm on tour for a month in Germany with a circus, wish I could play with it now... Is there a German distributor that will have it before the end of the month ?
 
i was just giving it a quick try (the SD wav example), the file opens alright but can't seem to get it outputting sound. i have a working setup (wm8731) with the old i2s/DMA library, the only difference being i have I2S0_TX_FS on pin 4 and I2S0_TXD0 on pin 3.

in the example sketch, i've changed audioShield to:

Code:
AudioControlWM8731 audioShield;

and in audio.cpp, i've changed, in AudioOutputI2S::begin :

Code:
//CORE_PIN22_CONFIG = PORT_PCR_MUX(6); // pin 22, PTC1, I2S0_TXD0

CORE_PIN3_CONFIG = PORT_PCR_MUX(6); // pin 3, PTC1, I2S0_TXD0

and in AudioOutputI2S::config_i2s :

Code:
//CORE_PIN23_CONFIG = PORT_PCR_MUX(6); // pin 23, PTC2, I2S0_TX_FS (LRCLK)
//CORE_PIN9_CONFIG  = PORT_PCR_MUX(6); // pin  9, PTC3, I2S0_TX_BCLK
//CORE_PIN11_CONFIG = PORT_PCR_MUX(6); // pin 11, PTC6, I2S0_MCLK

CORE_PIN4_CONFIG = PORT_PCR_MUX(6); // pin 4, PTC2, I2S0_TX_FS (LRCLK)
CORE_PIN9_CONFIG  = PORT_PCR_MUX(6); // pin  9, PTC3, I2S0_TX_BCLK
CORE_PIN11_CONFIG = PORT_PCR_MUX(6); // pin 11, PTC6, I2S0_MCLK

but that should be it, no?

thanks!
 
Is it possible to use I2C0 to control another chip at the same time as you're playing audio without interfering with it?

I'd like to drive some leds while playing audio using this IC, without having to resort to breaking out the pins for the second I2C port on the underside of the teensy:
http://www.adafruit.com/products/1427
 
Oh, cool, I see now... you used the Wire library. I've never used I2C so I didn't know it was a standard Arduino lib and I was concerned if I tried to write something to to the bus it would collide with data being sent in some interrupt.

How about SPI? Are there any precautions that need to be taken if you wish to use that? With the WaveHC lib I had to disable interrupts before I attempted to access the card to see if the next file I wanted to play existed before stopping the current one. I'm sure that particular issue doesn't exist with your library, but what if I were to connect another SPI device to the bus? Could I safely transfer data to it? It appears the Due added some new SPI functions for handling multiple slaves but does that all just work when an interrupt calls it while you're in the middle of a transfer?
 
Speaking of I2C and the Audio add-on shield, does the add-on board have the 4.7K pull-up resistors for I2C? If so, if I have other I2C devices, would I still need my own pull-up resistors? If it doesn't have the resistors, I assume we would need to add them to talk to the audio components?
 
Last edited:
Speaking of I2C and the Audio add-on shield, does the add-on board have the 4.7K pull-up resistors for I2C?

Yes, it has 2K pullup resistors on SDA and SCL.

If so, if I have other I2C devices, would I still need my own pull-up resistors?

The 2K resistors on the audio board ought to be plenty if you connect other I2C devices.


I should mention the actual audio data is sent using I2S, which is entirely different than I2C, even though the first 2 letters are the same.

(edited, typo fixed)

I2C uses 2 signals: SDA and SCL.

I2S uses 5 signals: 3 clocks (LRCLK, BCLK, MCLK) and 2 data pins (TX, RX).


The I2C and I2S ports are completely independent and can work simultaneously.
 
Last edited:
With the WaveHC lib I had to disable interrupts before I attempted to access the card to see if the next file I wanted to play existed before stopping the current one. I'm sure that particular issue doesn't exist with your library, but what if I were to connect another SPI device to the bus?

Actually, this same issue exists when you are playing a .WAV file. The library is regularly using the SPI port to talk to the SD card.

You can use the SPI port while audio is played from other sources, like streaming the input to output (possibly through other objects that modify the data). The audio library doesn't use the SPI port for normal operations, but it does need it for accessing the SD card.


Could I safely transfer data to it? It appears the Due added some new SPI functions for handling multiple slaves but does that all just work when an interrupt calls it while you're in the middle of a transfer?

Perhaps a future version will provide a way to negotiate access to the SPI while playing from the SD card. But that is quite tricky. If the audio library needs to obtain another block of audio from the SD card, it really can't wait.
 
i was just giving it a quick try (the SD wav example), the file opens alright but can't seem to get it outputting sound.

You might check your .wav file. Currently the library only supports 16 bit, 44.1 kHz MONO wav files. I'll add support for stereo format and upconverting some of the lower sample rates. But in this first beta release, only the simplest format is supported.
 
You might check your .wav file. Currently the library only supports 16 bit, 44.1 kHz MONO wav files. I'll add support for stereo format and upconverting some of the lower sample rates. But in this first beta release, only the simplest format is supported.

i think it must be something else, the 440 hz sine example (miditone) doesn't work either, unfortunately. i'm trying to figure out what's going on, but can't seem to figure out what "AudioConnection" is doing, or where it's coming from?
 
I2S uses 2 signals: SDA and SCL. I2S uses 3 clocks (LRCLK, BCLK, MCLK) and 2 data pins (TX, RX). The I2C and I2S ports are completely independent and can work simultaneously.

Normally I wouldn't point out minor mistakes like this, but I've never used I2S or I2C before and I'm not sure which is which from this...
 
How does the audio mixing work? I see an audio mix function that appears to mix up to four channels. The output right now is mono, correct? So that would be four mono 44,100hz wav files you can play at once?
What happens if you attempt to play another file if four are already playing? Does it simply not play?

I presume for now, the volume of each channel is simply divided by 4 before being output? It would be nice eventually to have automatic gain control. Compression would be nice too.
 
Normally I wouldn't point out minor mistakes like this, but I've never used I2S or I2C before and I'm not sure which is which from this...

What he meant to say was:
"I2C uses 2 signals: SDA and SCL. I2S uses 3 clocks (LRCLK, BCLK, MCLK) and 2 data pins (TX, RX)."
 
Opps, yeah, bad typo. I edited the post above.

I2C is a fairly slow bus (100 or 400 kbps) used to control chips. The audio board uses it to let you control settings, like the output volume, which input is selected (line in vs mic), and lots of other optional features. I2C is a "bus", meaning many different chips can be connected to the same 2 wires. You can connect other I2C chips, as long as they don't use the same address. Communication on I2C is in bursts, typically short messages, with the 2 wires idle between messages.

I2S is a fairly fast (1.4 Mbit/sec) digital audio protocol. Unlike I2C, I2S is meant as a point-to-point protocol. The BCLK (bit clock) signal indicates when each TX and RX signals has a new bit of data. The LRCLK (left-right clock) signal indicates which bits are the actual 16 bit words. When LRCLK is high, the left channel is sending, and when it's low, the right is sending (actually, there's a 1 clock delay.... minor details). MCLK (master clock) technically isn't part of I2S. It's simply a clock at 256 times the speed of LRCLK, or 11.29 MHz. Almost all audio chips need MCLK.
 
How does the audio mixing work? I see an audio mix function that appears to mix up to four channels.

Yup, pretty much just like that, it accepts up to 4 inputs. The streams are simply added together and output.

The output right now is mono, correct?

The WAV file player currently only parses mono WAV files. I'll add stereo parsing soon.

But the library itself supports any number of streams, within CPU and memory limits.

So that would be four mono 44,100hz wav files you can play at once?

Yes, if your SD card (and the SD library) can keep up.

In 1.18-rc2, I added a speedup to the SD library. So far, I haven't retested its performance with the audio stuff. But before the speedup, I was seeing about 40% CPU usage for reading a single mono stream from the SD card, mostly due to slowness in the SD library.

What happens if you attempt to play another file if four are already playing? Does it simply not play?

Each instance of the WAV player is independent. If you create 4 WAV player objects and decide you want to play a 5th file, how you do that is up to you. If your code has only created 4 of those objects, obviously you're going to need to stop one of them when you want to start playing a 5th file. The reason the limit is 4 is only because you created 4 independent objects.

Of course, you could just create 5 WAV player objects. The mixer supports only 4 inputs, but don't let that stop you! Just create 2 mixes and feed the output of one into the input of another, and now you can mix 7 channels!

I presume for now, the volume of each channel is simply divided by 4 before being output?

Oh, looks like I commented out the gain function. I'll put that back in. Remember, this stuff is very "beta" right now.....

The mixer does support variable gain on each channel. It defaults to a gain of 1 on each channel. If your mix exceeds the maximum level, the output clips, just like real audio gear (well, except this is digital audio, so there's a hard clipping limit rather than a lot of extra headroom like you'd find on analog gear).

It would be nice eventually to have automatic gain control. Compression would be nice too.

Pull requests are welcome!

In a week or two, I'll write up a how-to guide about creating your own audio effects. But here's a start.....

Start by finding a similar object in Audio.h and copy its definition, and of course change the name. All objects much inherit from AudioStream. If your object has inputs, you must have "audio_block_t *inputQueueArray[num]" in the private section, where "num" is the number of inputs. Your constructor must initialize AudioStream with the number of inputs and that array. You also must have the virtual update() function in your public section. Everything else is optional... just add private variables as needed for whatever you want to do, and add public functions for whatever stuff you want to be accessible from the Arduino sketch. The simplest way to get this "boilerplate" stuff right is to copy an object definition that already works. Maybe I ought to publish a template example?

In your actual code, the only function you must implement is update(). If you don't, the compiler will give a rather unhelpful vtable error.

If your object gets input, in update() you'll call receiveReadOnly(channel) or receiveWritable(channel) to acquire any incoming audio. The library allows shared memory, so receiveReadOnly() is more efficient in if the same audio is fed to other objects. Each input can only source 1 block, so you only need to call either of these one for each input channel. As the name suggests, you choose which function depending on whether you will modify the contents of the audio data. The mixer object, for example, calls receiveWritable() for the first block it acquires, and then receiveReadOnly() for any others. These functions can and do return NULL if there is no input, so you must check for NULL and treat the input as silent.

If you need to create audio, or you just need another block to fill, call allocate(). It too can return NULL, so you must check.

When you obtain audio from allocate(), receiveReadOnly() or receiveWritable(), you own it. You must call release() to free the memory. If you need to buffer audio, you can store block pointers in your object's private variables and use them on future update() runs, but this consumes the precious audio memory, so only keep blocks allocated if you really must.

The pointers you get, type "audio_block_t *" are a struct with 1 member you're mean to access: "data", which is an array of 128 int16_t's with the actual audio data. The struct has a couple other members which you should not touch. If you obtained the block with receiveReadOnly(), do not write to the data[] array. But you still must call release() when you're done with that block.

Once you've got your input blocks, plus any new blocks you need, do whatever your object will do. If you're modifying audio, you'll probably obtain the inputs as writeable blocks and change their data[] arrays directly. If you're synthesizing something, you'll probably get new blocks with allocate() and fill them up. Of course, you can do anything you want, within the limits of your programming skill and the available memory and CPU power.

If your object has outputs, call transmit(block, channel) to send audio out. It's ok to transmit the same block to multiple outputs. You still own that block, even after transmit. You must always call release() for every block you obtain with receiveReadOnly(), receiveWritable() or allocate().

For example, if you wanted to create automatic gain control, you might have 1 input and 1 output. You'd call receiveWritable(), then perhaps compute an average by just summing up the samples (inverting the negative ones, of course). Or maybe you'd square them and add them all up? Then you might adjust your gain slightly if the average is above or below the target, which of course would be a private member variable, and multiply all 128 samples by the gain setting. Then just transmit and release the block.

Every objects gets simple CPU usage tracking (but I need to publish an example), so you can test how much CPU you're using. The Cortex-M4 DSP optimizations can really help. In this AGC example, to compute the average, you could fetch the input samples 2 at a time using a 32 bit pointer. The M4 optimizes multiple reads in a row, where a normal 32 bit read takes 2 cycles, but subsequent back-to-back reads take only 1 more cycle. So you could fetch 8 inputs in only 5 cycles. The DSP instructions feature a dual 16x16 multiply-and-accumulate (allowing inputs from separate halves of 32 bit registers), so you could square and sum each pair of inputs in just 1 cycle. The average could be computed in just 144 cycles without looping overhead. To keep things in perspective, ALL the update functions for all audio objects must complete in under 278528 cycles. The CPU usage functions can tell you what fraction of that total you're using up.

Likewise, the AudioMemory() function at the beginning of setup() creates the pool of memory which provides all audio_block_t data. There are functions to query the current and worst case usage, so you can tell if you've got memory issues. But if you receive or allocate audio blocks and fail to release them, you'll quickly run out of memory and the entire system will go silent, so memory leaks are pretty obvious.

Objects that actually move data on or off the chip have some other requirements, which are what causes every update() function to actually run every 2.9 ms. But you don't need to worry about those if your object runs entirely on-chip, using only receiveReadOnly(), receiveWritable(), allocate(), transmit() and release().
 
I've added support for stereo format WAV files. They must be 16 bit, 44.1 kHz, but now either mono or stereo should play.

The 1.18-rc2 SD library speedup is working. WAV file playing now consumes approx 12% CPU, down from 40% before the speedup.
 
further re using this with wm8731.

i've got the SD card example working now. strangely, it plays the file only if/once i open the serial monitor? the sine example still doesn't work for me.

as an aside, using AudioControlWM8731 gives me:

error: cannot declare variable 'codec' to be of abstract type 'AudioControlWM8731'

so i added

Code:
bool disable(void) { return false; }

in class AudioControlWM8731 : public AudioControl , which seemed to be missing?
 
Sorry, that WM8731 code probably has a lot of bugs. I switched to the SGTL5000 fairly early in the library's development and haven't tested with the WM8731 since.

Pull requests are welcome....
 
4x speedup! Not bad.

Is that for a single wav file? Mono or stereo?

Yes, that's a single WAV file.

The library runs every 2.9 ms, which is 128 audio samples. While playing a mono WAV file, it ends up using 12% every other update and only 1% on the other half. While playing stereo, it uses 12% every time.

It would be ideal if the usage were 6% every time on mono mode, but that's simply not how the SD library works. It reads the SD card 512 bytes at a time, since the card has 512 byte sectors. In mono mode, one 512 byte sector provides 256 audio samples, which is enough for 2 update cycles, so the WAV player object calls the SD library every other update. In stereo mode, every update consumes an entire 512 byte sector.
 
Sorry, that WM8731 code probably has a lot of bugs. I switched to the SGTL5000 fairly early in the library's development and haven't tested with the WM8731 since.

Pull requests are welcome....

no worries, i don't think it's anything essential. sounding very nice, in fact. thank you very much.

i'm still puzzling over SD card example though, and must be missing something obvious? i'm simply trying to loop a file but it's only ever playing once. then things stop entirely:

Code:
void loop() {
   
   Serial.println("play");
   wav.play("1.wav");
   delay(4000);
   wav.stop();
   Serial.println("stop");
   delay(20);
}

the rest is essentially playwavfromsdcard
 
Status
Not open for further replies.
Back
Top