Audio Library

[MickMad]

Hey there, I've just tested the audio lib with the Mikroe proto board and the new PlayMidiTones sketch (which actually just plays sinewaves).

While reading through the code I discovered that the lib assumes that the ADC/DAC/CODEC on the I2S channel is in Slave mode, so I've added a #define and some #if blocks in the AudioI2S*put classes code to check wether the MCLK signal is external or internal and to set the appropriate registers to signal that.

in Audio.h, line 3

#define MCLK_IS_EXTERNAL 1

in Audio.h, line 337

	bool disable(void) { return false; }

in Audio.cpp, line 631

void AudioOutputI2S::config_i2s(void)
{
	SIM_SCGC6 |= SIM_SCGC6_I2S;
	SIM_SCGC7 |= SIM_SCGC7_DMA;
	SIM_SCGC6 |= SIM_SCGC6_DMAMUX;

	// if either transmitter or receiver is enabled, do nothing
	if (I2S0_TCSR & I2S_TCSR_TE) return;
	if (I2S0_RCSR & I2S_RCSR_RE) return;
#if MCLK_IS_EXTERNAL
	// Select input clock 0
	// Configure to input the bit-clock from pin, bypasses the MCLK divider
	I2S0_MCR = I2S_MCR_MICS(0);
	I2S0_MDR = 0;
#else
	// enable MCLK output
	I2S0_MCR = I2S_MCR_MICS(3) | I2S_MCR_MOE;
	I2S0_MDR = I2S_MDR_FRACT(1) | I2S_MDR_DIVIDE(16);
#endif
	// configure transmitter
	I2S0_TMR = 0;
	I2S0_TCR1 = I2S_TCR1_TFW(1);  // watermark at half fifo size
	I2S0_TCR2 = I2S_TCR2_SYNC(0) | I2S_TCR2_BCP;
#if !MCLK_IS_EXTERNAL
	I2S0_TCR2 |= I2S_TCR2_MSEL(1) | I2S_TCR2_BCD | I2S_TCR2_DIV(3);
#endif
	I2S0_TCR3 = I2S_TCR3_TCE;
	I2S0_TCR4 = I2S_TCR4_FRSZ(1) | I2S_TCR4_SYWD(15) | I2S_TCR4_MF
		| I2S_TCR4_FSE | I2S_TCR4_FSP;
#if !MCLK_IS_EXTERNAL
	I2S0_TCR4 |= I2S_TCR4_FSD;
#endif
	I2S0_TCR5 = I2S_TCR5_WNW(15) | I2S_TCR5_W0W(15) | I2S_TCR5_FBT(15);

	// configure receiver (sync'd to transmitter clocks)
	I2S0_RMR = 0;
	I2S0_RCR1 = I2S_RCR1_RFW(1);
	I2S0_RCR2 = I2S_RCR2_SYNC(1) | I2S_TCR2_BCP;
#if !MCLK_IS_EXTERNAL
	I2S0_RCR2 |= I2S_RCR2_MSEL(1) | I2S_RCR2_BCD | I2S_RCR2_DIV(3);
#endif
	I2S0_RCR3 = I2S_RCR3_RCE;
	I2S0_RCR4 = I2S_RCR4_FRSZ(1) | I2S_RCR4_SYWD(15) | I2S_RCR4_MF
		| I2S_RCR4_FSE | I2S_RCR4_FSP | I2S_RCR4_FSD;
#if !MCLK_IS_EXTERNAL
	I2S0_RCR4 |= I2S_RCR4_FSD;
#endif
	I2S0_RCR5 = I2S_RCR5_WNW(15) | I2S_RCR5_W0W(15) | I2S_RCR5_FBT(15);

	// configure pin mux for 3 clock signals
	CORE_PIN23_CONFIG = PORT_PCR_MUX(6); // pin 23, PTC2, I2S0_TX_FS (LRCLK)
	CORE_PIN9_CONFIG  = PORT_PCR_MUX(6); // pin  9, PTC3, I2S0_TX_BCLK
	CORE_PIN11_CONFIG = PORT_PCR_MUX(6); // pin 11, PTC6, I2S0_MCLK
}

in Audio.cpp, line 1684

bool AudioControlWM8731::enable(void)
{
	Wire.begin();
	delay(5);
	//write(WM8731_REG_RESET, 0);
#if MCLK_IS_EXTERNAL
	write(WM8731_REG_INTERFACE, 0x42); // I2S, 16 bit, MCLK master
#else
	write(WM8731_REG_INTERFACE, 0x02); // I2S, 16 bit, MCLK slave
#endif
	write(WM8731_REG_SAMPLING, 0x20);  // 256*Fs, 44.1 kHz, MCLK/1

in Audio.cpp, line 1702

	write(WM8731_REG_POWERDOWN, 0x00); // codec powerdown

This last line is important, without it the WM8731 wouldn't start properly when activated.

It works, though there was a missing mysine.amplitude() call in the demo sketch and I lost half an hour just to find that The pins are exactly the same as the Audio Adaptor, minus MCLK which is not connected:

Audio Codec PROTO SCK -> Teensy 9 (I2S0_BCLK)

Audio Codec PROTO ADCL -> Teensy 12 (I2S0_RX_FS)
Audio Codec PROTO MISO -> Teensy 13 (I2S0_RXD0)

Audio Codec PROTO MOSI -> Teensy 22 (I2S0_TXD0)
Audio Codec PROTO DACL -> Teensy 23 (I2S0_TX_FS)

I've also noticed that the output frequency is slightly higher than the 440 Hz it should output.

I still gotta test the codec in input, will let you know.

I will probably make a pull request tomorrow, is it okay?

[PaulStoffregen]

The trouble with MCLK as an input is the timing won't necessarily be synchronous with the CPU. It's probably not a big deal if you don't use another other input or output objects that move audio on/off chip. But if you do, things can drift out of sync. I'm not even sure what will happen... if the result might be as benign as infrequent 3ms dropouts, or if the whole thing might crash hard?

[PaulStoffregen]

I've been working on converting 11025 and 22050 Hz samples by linear interpolation. It seems to be working pretty well, but admittedly I've only been listening pretty casually, not comparing to the 44100 version (yet). I have one minor bug to fix in the utility that converts WAV to C arrays you can drop into your sketch, and then I'm going to commit this to github.

8000 Hz is the real problem. Lots of WAV files are 8 kHz sampled. Especially for embedded them into the extra on-chip flash memory, you get a lot more playback time, but of course with reduced bandwidth.

This is probably going to be a bit controversial, but my current plan is to convert every 2 samples at 8000 Hz to 11 samples at 44000 Hz (using linear interpolation), and then feed that wrong-rate 44000 Hz data out at 44100 Hz. That's a 0.2% error. 8000 Hz files (stored in flash) will actually play at 8018 Hz. It's not technically correct, but this is a trade-off in programming time, to let me publish something that will work ok for most people who just want to play sound effects with easy conversion from already-made WAV files, and let me move on to other parts of the library.

Anyone who feels strongly about correct pitch can convert their data to 8018 Hz. Or if they _really_ feel strongly, of course well designed pull requests are always welcome.

[PaulStoffregen]

Currently I've been working with 6 small WAV files, which will be embedded in a new example program.

If anyone has any favorite (short) audio clips that are freely sharable, please post them. My goal is to keep the entire collection of WAVs in the example to about 100K, so the code can compile for either Teensy 3.0 or 3.1.

[Tomsim]

Thanks Paul. I added unused PWM output object per your suggestion and I can get audio sample data now using AudioInputAnalog. Next, I'll try AudioInputAnalog with your FFT example.

[mxxx]

I'm trying for linear interpolation today, pretty much like this at the moment.

        } else if (rate == 22050) {
                int16_t s0, s1, s2, s3, s4;
                s0 = prior;
                if (bits == 8) {
                        for (i=0; i < AUDIO_BLOCK_SAMPLES; i += 8) {
                                tmp32 = *in++;
                                s1 = (int8_t)((tmp32 >> 0) & 255) << 8;
                                s2 = (int8_t)((tmp32 >> 8) & 255) << 8;
                                s3 = (int8_t)((tmp32 >> 16) & 255) << 8;
                                s4 = (int8_t)((tmp32 >> 24) & 255) << 8;
                                *out++ = (s0 + s1) / 2;
                                *out++ = s1;
                                *out++ = (s1 + s2) / 2;
                                *out++ = s2;
                                *out++ = (s2 + s3) / 2;
                                *out++ = s3;
                                *out++ = (s3 + s4) / 2;
                                *out++ = s4;
                                s0 = s4;
                        }

This approach requires keeping only 1 prior sample, but 2 or 3 prior samples could be kept. Those 8 output lines could become simple polynomials implementing a short FIR filter. But what equations to use?

pd tabread4~, for example, defaults to 4 point polynomial / Lagrange (-1, 0 ,1 , 2), so something like this might be worth considering then (ie, if 3 samples could be kept)? will "rate" be an user-supplied argument or is it deriving from the file header?

[PaulStoffregen]

At the moment I'm parsing the WAV file's header and packing the audio into the array with the first 8 bytes as a header, indicating the sample rate, number of bits, and length.

Today I'm reconsidering the number of bits. I've been doing more listening tests with quality headphones, and I'm just not happy with 8 bits. It sounds terrible on my PC and on Teensy3. Some clips encode ok, but many end up with considerable "hiss" noise.

This morning I tried 8 but mu-law encoding, so far only on my PC, and it sounds a LOT better. Since the files get translated with code I'm writing, I'm considering switching from linear 8 bit to mu-law. It will add another day of coding, but in the end I believe it will really help make sound clip playing projects sound much better with the limited 256K flash.

[scswift]

What's this 8 bit stuff about? I thought the library played 16 bit wavs. Does it downconvert them to 8 bit or are you just talking about a way to get more out of 8 bit clips stored in flash?

[PaulStoffregen]

I've uploaded the latest Audio library code to Github.

https://github.com/PaulStoffregen/Audio

There's now a PlayFromSketch example with 6 sound clips. It's capable of playing all 6 simultaneously (6 voices).

The audio library is still fully 16 bits. The new AudioPlayMemory object internally converts from 8 to 16 bits as it reads from the on-chip memory, if the stored copy is 8 bit. This is only to allow longer clips to be stored in the limited (approx 220K) on-chip memory.

In the PlayFromSketch example's directory is a wav2sketch program that converts 16 bit WAV files to the data arrays for use with the AudioPlayMemory. It can convert 2 different ways, either original 16 bit PCM, or 8 bit u-law encoding. The default is 8 bit u-law, which sounds pretty good for many files, but it does add distortion that's audible with some types of sounds. Just run "wav2sketch -16" to convert in 16 bit mode. This obviously uses twice as much flash, but you get a perfect lossless copy of the original WAV file. The audio library can play either type. I put a mix of them in the example.

Only sample rates 44100, 22050 and 11025 are supported. 22050 and 11025 are upconverted to 44100 using simple linear interpolation. @mxxx, how would that 4 point polynomial fit into this code? Certainly more than just "s0" could be kept between runs, if needed.

[mxxx]

Only sample rates 44100, 22050 and 11025 are supported. 22050 and 11025 are upconverted to 44100 using simple linear interpolation. @mxxx, how would that 4 point polynomial fit into this code? Certainly more than just "s0" could be kept between runs, if needed.

i wish i knew enough of this stuff to be of any help. i think it's mostly about using more expensive forms of interpolation. this (or similar) is how interpolation is or used to be done in said pd object:

 


      cminusb = c-b;
      *out++ = b + frac * (
                     cminusb - 0.1666667f * (1.-frac) * (
                                                 (d - a - 3.0f * cminusb) * frac + (d + 2.0f*a - 3.0f*b)
                                                               )
                           );

where a-d are points -1, 0, 1, and 2; frac the fractional part of an index into the sample array. i'm not sure whether it would be worth the trouble though, considering AudioPlayMemory is (i'd guess) going to used only for lo-fi stuff anyways. it might be more relevant for some kind of AudioPlaySDcardWAV::varispeedplay(...) type thing.

[PaulStoffregen]

I am planning to bring this upsampling code into to AudioPlaySDcardWAV, so files at those slower sample rates can play from SD cards. I'm also planning to work on another function to begin playing with an optional delay, where the library will use the delay to attempt to schedule multiple instances of AudioPlaySDcardWAV to interleave their reading from the SD card. 44.1 kHz 16 bit stereo requires reading a sector every update, but mono and slower rates don't need a read every time. If interleaved properly, more simultaneous streams ought to be able to play.

For now, I think the linear interpolation sounds pretty good. Even with good quality headphones, I can't hear much difference between the Teensy output and my PC's output, but then my PC might be using a less than ideal upsampling algorithm too. For now, I'm moving on to other stuff, like an object to output to Teensy 3.1's on-chip DAC. I'll revisit this if one the gurus like Adrian Freed chimes in with a compelling algorithm....

[Nantonos]

8000 Hz is the real problem. Lots of WAV files are 8 kHz sampled. Especially for embedded them into the extra on-chip flash memory, you get a lot more playback time, but of course with reduced bandwidth.

This is probably going to be a bit controversial, but my current plan is to convert every 2 samples at 8000 Hz to 11 samples at 44000 Hz (using linear interpolation), and then feed that wrong-rate 44000 Hz data out at 44100 Hz. That's a 0.2% error. 8000 Hz files (stored in flash) will actually play at 8018 Hz.

That seems totally fine to me. They are slightly pitch shifted but not by much: 3.9 cents sharp (there are 100 cents in a semitone). For a format that is prioritising space over fidelity and is likely going to be used for sound effects and so on rather than music, that seems totally fine.

If someone wanted better conversion, they could do it on a computer (and if they wanted better quality they wouldn't be starting with 8k WAV files).

[PaulStoffregen]

Today I've been working on supporting Teensy 3.1's built-in DAC in the audio library. I currently have it running with interrupts and I hope to convert to DMA soon.

I have a bit of excellent news to share. It turns out the DAC's bandwidth is much better than Freescale's datasheet specifies: 15 us typical, 30 us worst case.

But if you check out this scope capture, the rise time is approx 0.6 us. While this measurement isn't precise enough for checking settling to 0.8 mV, visually it looks like the waveform setting pretty well in about 1 to 2 us.


(click for larger)

As you can see, it's easily capable of creating a 22 kHz square wave. The 30 us worst case was a bit slow, but it turns out the hardware is at about 10X faster than the datasheet says, so full bandwidth audio won't be any problem at all.

[PaulStoffregen]

I got the DAC working! Code is now on Github.

https://github.com/PaulStoffregen/Audio

To use it, you'll also need to update mk20dx128.h for the extra DMA channel definitions. The file is here:

https://github.com/PaulStoffregen/co...y3/mk20dx128.h


Edit: The update scheduling doesn't yet work on the DAC object, so you MUST have one of the I2S objects or the PWM output objects. I'll work on this bug soon...

[PaulStoffregen]

While reading through the code I discovered that the lib assumes that the ADC/DAC/CODEC on the I2S channel is in Slave mode, so I've added a #define and some #if blocks in the AudioI2S*put classes code to check wether the MCLK signal is external or internal
.....
I will probably make a pull request tomorrow, is it okay?

If you're still working on this, I would like a pull request, but done a bit differently.

Instead of a #define that needs to be edited within the library source, I'd like to see 2 new objects with slightly different names. Perhaps AudioOutputI2Sslave and AudioInputI2Sslave? They should share the same dma_ch0_isr() and dma_ch1_isr() and reuse all the same static variables and buffers.

Perhaps the cleanest way to do this might be to have those 2 new classes inherit from the 2 existing ones? You'd probably only need to implement a different begin() and config_i2s(), and let the C++ inheritance reuse everything else. It's fine to change any of the I2S private members to protected, if necessary.

If that doesn't work out, the new slave-mode classes could just be normal audio objects that inherit from AudioStream. You'd need to add friend class lines into the existing I2S objects to allow these new ones to access their private variables.

I'd like to see it done this way, as 2 new classes, so casual Arduino users can simply add the extra word onto the end of the object name to use your slave-mode version. They shouldn't need to edit a #define within the library.

[MickMad]

If you're still working on this, I would like a pull request, but done a bit differently.
I'd like to see it done this way, as 2 new classes, so casual Arduino users can simply add the extra word onto the end of the object name to use your slave-mode version. They shouldn't need to edit a #define within the library.

That is exactly what I want to avoid too I'm working on the mods right now.

[daperl]

@Paul

In the PassThroughAnalog example you don't declare audioInput.

[scswift]

// slowly ramp up to DC voltage, approx 1/4 second
        for (int16_t i=0; i<128; i++) {
                analogWrite(A14, i);
                delay(2);
        }

The DAC is 12bit, so why are you ramping up to 127 instead of 2047 here? You're not giving up 4 bits of accuracy are you?

Hm, I see 2047 in some other functions after that but I don't see analogWrite or analogWriteResolution anywhere else... I guess you're setting the resolution after that somehow and outputting the data to the DAC at 12 bit accuracy using some register somewhere.

[PaulStoffregen]

The DAC is 12bit, so why are you ramping up to 127 instead of 2047 here?

Because analogWrite defaults to 8 bit

You're not giving up 4 bits of accuracy are you?

Only during the initial ramp up to the DC offset.

Hm, I see 2047 in some other functions after that but I don't see analogWrite or analogWriteResolution anywhere else... I guess you're setting the resolution after that somehow and outputting the data to the DAC at 12 bit accuracy using some register somewhere.

Don't worry. Once it starts running, the DMA transfers move 12 bit data.

In fact, this part of the code translates the 16 bit audio from the library to the 12 bits needed by the DAC.

        block = AudioOutputAnalog::block_left_1st;
        if (block) {
                src = &block->data[offset];
                do {
                        // TODO: this should probably dither
                        *dest++ = ((*src++) + 32767) >> 4;
                } while (dest < end);

You can see the conversion from the library's signed integers to the DAC's unsigned range, and the 4 bit shift that converts from 16 to 12 bits.

Someday this code might get proper dithering, as the comment indicates. For now, the conversion is done by simply discarding the 4 lowest bits.

[PaulStoffregen]

The AudioOutputAnalog update scheduling bug is fixed. You can now use Teensy 3.1's DAC without I2S or any other input or output objects.

I also just fixed the mixer gain function. You can set the gain on each mixer channel, using myMixer.gain(channel, gain). The gain range is 0.0 to 32767.0 (floating point), so you can also use it to amplify signals. The default gain is 1.0.

[scswift]

You've got the DAC set up to use the 3.3v internal regulator as a reference, right?

I was reading this page, and came across the following:
https://pjrc.com/teensy/teensy31.html

The output is created by the stable reference voltage, so it's doesn't vary if your power supply voltage changes slightly.

It got me wondering.

If the DAC uses 3.3v as its reference, what happens when one decides to power their device from an external 3.3v regulator thus bypassing the internal regulator, and said regulator is supplying other power hungry devices? The 3.3v output would no longer be stable, would it not? For that matter, isn't the power hungry SD card powered from the internal 3.3v regulator anyway? Wouldn't that be introducing a lot of noise on the output?

It seems like it would be better to use the internal 1.2v reference for the DAC when playing audio though it. That would remain stable even if the 3.3v line itself isn't particularly stable, right? It's also close to consumer line level which is .894v peak to peak. One could potentially get away without using a voltage divider.

I only mention this because on my own boards I had major issues with the audio output due to attaching a couple TLC5947 LED drivers to it. They were introducing so much noise into my audio system that I had to use a ground loop isolator when attaching my board to an amplifier. My problem may have been an issue with noise on my system's ground rather than noise on the 5v reference for my DAC though. I was told that if I referenced my voltage divider to the amplifier ground instead of grounding it to both my board and the amplifier that that would fix it and cancel the noise out. I don't know for sure though as I have yet to test it.

[PaulStoffregen]

It seems like it would be better to use the internal 1.2v reference for the DAC when playing audio though it.

Yes, that's a good point.

I've added an analogReference() function to the AudioOutputAnalog object. You can use it like this:

  dac.analogReference(INTERNAL);

It does result in a sudden jump on the DC level when changing reference voltage. At some point I'll add code to deal with that issue.....

[cartere]

The AudioOutputAnalog update scheduling bug is fixed. You can now use Teensy 3.1's DAC without I2S or any other input or output objects.

I also just fixed the mixer gain function. You can set the gain on each mixer channel, using myMixer.gain(channel, gain). The gain range is 0.0 to 32767.0 (floating point), so you can also use it to amplify signals. The default gain is 1.0.

Thanks Paul, I have a proof of concept running with a kludge this will clean things up

[psxpetey]

teensy you are the coolest

[mattomatto]

Hi there,

I have already developed audio code that I would like to just output samples through the teensy audio shield. Is the best way to do this by creating my own audio object in the library and using the following format?:

void audioObjectTemplate::update(void)
{
	if (magnitude > 0 && (block = allocate()) != NULL) {
		// Declare some variables
		for (i=0; i < AUDIO_BLOCK_SAMPLES; i++) {
			//Generate and process audio samples
		}
		//update variables
		transmit(block);
		release(block);
	} else {
		//What should/can I do in here?
	}
}

Thanks