Audio Library

saxen · Feb 24, 2014

Hi Paul,
is there a reason why you reset the "aprev" and "bprev" variables every AUDIO_BLOCK_SAMPLES in the biquadratic filter?
Doing in this way, I have some problem with the filter initial condition every audio block coming. It could be fixed by making "aprev" and "bprev" as object variable, and initialize them once in an init() method.

This is the input waveform:

This is the original output:

This is after my changes:

Thank you,
Enrico

Rossitron · Feb 24, 2014

PaulStoffregen said:
Any ideas how this could be done better? I'd really like it to automatically find and subtract the DC level, so people can use a couple resistor and a capacitor and the code will adapt automatically to their DC bais.

I really like it being able to automatically find the DC offset too. Very fast run-n-gun arduino style.

PaulStoffregen said:
Maybe an initial fast response to get the approx DC level, followed by a much slower response afterward?

Muuuch slower. Being able to handle clipping (asymmetric too) would make it usable for a microphone or any other potentially sloppy single supply input

What if the offset was public and in AudioInputAnalog::begin if it's not zero (has been set by hand outside of the lib) use that value and don't auto find the offset. I think that would keep most people happy. Auto should work for most people, and when you need to set it by hand it wont be a change to the lib (not very arduino style).

robsoles · Feb 24, 2014

Hey Enrico, I just re-read the source to filter_biquad.* and formed the impression that aprev and bprev are (well, look like they should be being) preserved in the state array between calls so it shouldn't be necessary to make them persistent by other means afaict.

Would you please post your changes to filter_biquad.* (either to this thread or as a pull request to Paul) just to satisfy my curiosity?

saxen · Feb 24, 2014

robsoles said:
Hey Enrico, I just re-read the source to filter_biquad.* and formed the impression that aprev and bprev are (well, look like they should be being) preserved in the state array between calls so it shouldn't be necessary to make the persistent by other means afaict.

Would you please post your changes to filter_biquad.* (either to this thread or as a pull request to Paul) just to satisfy my curiosity?

Hi robsoles,
"aprev" and "bprev" are indeed saved in the state array,

Code:

*(state-2) = bprev;
*(state-3) = aprev;

however, in every update() call (every 128 samples) we also assign,

Code:

state = (int32_t *)definition;

Since definition[5] and definition[6] are 0, I assume "aprev" and "bprev" are lost at that point. Does this make sense to you?

Anyway, this is how I (quickly, maybe not so elegantly) changed the code...

Code:

void AudioFilterBiquad::update(void)
{
	audio_block_t *block;
	int32_t a0, a1, a2, b1, b2, sum;
//	uint32_t in2, out2, aprev, bprev, flag;
	uint32_t in2, out2, flag;
	uint32_t *data, *end;
	int32_t *state;

	block = receiveWritable();
	if (!block) return;
	data = (uint32_t *)(block->data);
	end = data + AUDIO_BLOCK_SAMPLES/2;
	state = (int32_t *)definition;
	do {
		a0 = *state++;
		a1 = *state++;
		a2 = *state++;
		b1 = *state++;
		b2 = *state++;
	//	aprev = *state++;
		*state++;
		*state++;
	//	bprev = *state++;
		sum = *state & 0x3FFF;
		do {
			in2 = *data;
			sum = signed_multiply_accumulate_32x16b(sum, a0, in2);
			sum = signed_multiply_accumulate_32x16t(sum, a1, aprev);
			sum = signed_multiply_accumulate_32x16b(sum, a2, aprev);
			sum = signed_multiply_accumulate_32x16t(sum, b1, bprev);
			sum = signed_multiply_accumulate_32x16b(sum, b2, bprev);
			out2 = (uint32_t)sum >> 14;
			sum &= 0x3FFF;
			sum = signed_multiply_accumulate_32x16t(sum, a0, in2);
			sum = signed_multiply_accumulate_32x16b(sum, a1, in2);
			sum = signed_multiply_accumulate_32x16t(sum, a2, aprev);
			sum = signed_multiply_accumulate_32x16b(sum, b1, out2);
			sum = signed_multiply_accumulate_32x16t(sum, b2, bprev);
			aprev = in2;
			bprev = pack_16x16(sum >> 14, out2);
			sum &= 0x3FFF;
			aprev = in2;
			*data++ = bprev;
		} while (data < end);
		flag = *state & 0x80000000;
		*state++ = sum | flag;
		*(state-2) = bprev;
		*(state-3) = aprev;
	} while (flag);
	transmit(block);
	release(block);
}

Code:

class AudioFilterBiquad : public AudioStream
{
public:
	AudioFilterBiquad(int *parameters)
	   : AudioStream(1, inputQueueArray), definition(parameters) { init(); }
	virtual void update(void);
	void init(){
		aprev=definition[5];
		bprev=definition[6];
	}
	void updateCoefs(int *source, bool doReset);
	void updateCoefs(int *source);
private:
	int *definition;
	uint32_t aprev, bprev;
	audio_block_t *inputQueueArray[1];
};

Thanks,
Enrico

robsoles · Feb 24, 2014

Thanks for showing us that Enrico.

state is being used as a copy of definition that can be moved without having to reset definition to its original condition after reading back 5 coefficients and aprev, bprev & sum, then processing samples, then writing back sum, bprev and aprev.

Without testing much and just working with assumptions: I think I will assume that *(state-2) and *(state-3) aren't interpreted properly by the compiler and they may be writing partway between actual pointers where *state++ moves the pointer appropriately but *(state-1) doesn't point at the same place that *--state; would (I think it might be pointing at the last byte of the previous int32_t rather than the first). I wonder if you would test this proposed fix on a 'vanilla' copy of the filter_biquad.cpp file from the library for me, I am meant to be doing something else

(am at work, for another 7 or so hours now!)

Code:

void AudioFilterBiquad::update(void)
{
	audio_block_t *block;
	int32_t a0, a1, a2, b1, b2, sum;
	uint32_t in2, out2, aprev, bprev, flag;
	uint32_t *data, *end;
	int32_t *state;

	block = receiveWritable();
	if (!block) return;
	data = (uint32_t *)(block->data);
	end = data + AUDIO_BLOCK_SAMPLES/2;
	state = (int32_t *)definition;
	do {
		a0 = *state++;
		a1 = *state++;
		a2 = *state++;
		b1 = *state++;
		b2 = *state++;
		aprev = *state++;
		bprev = *state++;
		sum = *state & 0x3FFF;
		do {
			in2 = *data;
			sum = signed_multiply_accumulate_32x16b(sum, a0, in2);
			sum = signed_multiply_accumulate_32x16t(sum, a1, aprev);
			sum = signed_multiply_accumulate_32x16b(sum, a2, aprev);
			sum = signed_multiply_accumulate_32x16t(sum, b1, bprev);
			sum = signed_multiply_accumulate_32x16b(sum, b2, bprev);
			out2 = (uint32_t)sum >> 14;
			sum &= 0x3FFF;
			sum = signed_multiply_accumulate_32x16t(sum, a0, in2);
			sum = signed_multiply_accumulate_32x16b(sum, a1, in2);
			sum = signed_multiply_accumulate_32x16t(sum, a2, aprev);
			sum = signed_multiply_accumulate_32x16b(sum, b1, out2);
			sum = signed_multiply_accumulate_32x16t(sum, b2, bprev);
			aprev = in2;
			bprev = pack_16x16(sum >> 14, out2);
			sum &= 0x3FFF;
			aprev = in2;
			*data++ = bprev;
		} while (data < end);
		flag = *state & 0x80000000;
		*state-- = sum | flag;
		*state-- = bprev;
		*state = aprev;
	} while (flag);
	transmit(block);
	release(block);
}

filter_biquad.h should be reset to its 'vanilla' state too please.

saxen · Feb 24, 2014

Thanks for your answer! I tried your code, but I get the same result as for the "vanilla" code.

I did few more testing, and I observed something strange.
- I print the value of the aprev and bprev of my modified version, together with those coming from the state array in your version (as well as in the vanilla). They are actually "most of the time" different.
- I print the address of state when I am reading aprev and bprev. The writing and the reading are actually in the same address, both with your code and the vanilla one (so the two things are interpreted in the same way). The strange thing (at least from my point of view) is that the address of "definition" seems to change! And when it changes, the "aprev" and "bpreb" of my version are different from those in the "state" array of the vanilla version.

--------------------
While I was writing this post, I found the solution. "Definition" was changing address because I was passing the same array of coefficients to different filters!!! If I create a different copy of the coefficients for each filter I do not have this problem anymore.

Wouldn't be better to avoid this thing? What do you think about my solution as an alternative?

Thank you,
Enrico

robsoles · Feb 24, 2014

I didn't know you were assigning the same 'parameters' array to more than instance of AudioFilterBiquad, I would have made a better assumption if I had known that.

My assumption was clearly wrong. A better assumption would be that the other instances of AudioFilterBiquad are 'mashing' the values and each instance must suffer from each other instances 'interference'.

If you want to use the AudioFilterBiquad that way, with more than one instance referring a single set of parameters, then your fix is best to that end but it might be better if you include 'sum' in your 'preserved' variables per instance.

I wouldn't personally 'push' that as any kind of 'fix' to the library because I don't think it is using the object as intended - I am fairly certain the intention is to have an individual set of parameters per instance of AudioFilterBiquad like I do in the example CalcBiquadToneControl.

If Paul (and other interested parties...) agree then perhaps it can be re-written to use only a five element parameter array holding the coefficients and the aprev, bprev & sum become static variables as you propose for aprev and bprev - I think this would be better but I hardly hate it the way it is.

saxen · Feb 24, 2014

Well it is not a big deal duplicating the coefficient set. However, biquadratic filters are usually meant to be used in cascade, and very often with the same set of parameters. That's why I would say it might be convenient.

Thanks again for your help, robsoles!

Enrico

robsoles · Feb 25, 2014

I didn't think hard enough about the specific behavior toward '*state' and use of 'flag' before. A single instance of AudioFilterBiquad can apply more than one set of coefficients.

Correct me if I'm wrong, Paul, but

Code:

int multipleFilters[]={lp_b0,lp_b1,lp_b2,lp_a1,lp_a2,0,0,0x80000000,hp_b0,hp_b1,hp_b2,hp_a1,hp_a2,0,0,0x80000000,nf_b0,nf_b1,nf_b2,nf_a1,nf_a2,0,0,0};

is how to set it up with the three filters lp, hp & nf

Of course I left my Teensy hardware at work so I can't immediately test this.

Am I wrong or did I miss a memo or were you keeping mum about this Paul?

Enrico, if I am not wrong you can cascade your filters in one AudioFilterBiquad object per channel and I need to rework .updateCoefs(..) and the tone control example too!

PaulStoffregen · Feb 25, 2014

Yes, I wrote the biquad filter to work as a cascade, but I haven't said anything about it since the one and only time I tried to test, it didn't seem to work. So I just committed the code and figured I'd get around to testing (and fixing) cascade mode another day.

I also don't know how you're supposed to compute the coefficients. For example, in a butterworth filter, some stages have higher Q so the whole thing end up maximally flat. In my dream world, where I could spend an infinite amount of time perfecting everything, I'd study up on the S parameter transformation stuff and create a javascript-based web page that computes any number of cascade parameters.

robsoles · Feb 25, 2014

I modified a local copy of .updateCoefs and my tone control example last night and tried it as soon as I hit the building at work; didn't work, no output at all as opposed mangling or anything silly like that - I love the idea so much that I am going to spend the next couple of chunks of spare time I get working on it coz a single instance applying all required filters, rather than instance per filter, is cooler.

I've an opinion about computing the coefficients but I think I will zip my lip (curb my fingers?) till I have read and reviewed more related stuff. I've been contemplating siblings for calcBiquad regarding butterworth and chebyshev filters, maybe if I pursue those enough I will have read and reviewed enough stuff...

robsoles · Feb 26, 2014

pretty sure I just committed the fix for multiple filter parameter set usage and made a pull request already.

Edit: Nope, fix didn't work - didn't make it worse but didn't make it better either; I'll keep poking it when/as I can.

PaulStoffregen · Feb 26, 2014

I've started the documentation. There is still a LOT of work left to do, but if anyone has time to look this over, I could really use some perspectives. Does it "work"? Have I missed stuff?

http://www.pjrc.com/teensy/td_libs_Audio.html

Also, if anyone has any ideas for the yet-to-be-written "Getting Started - Key Concepts" page, please speak up?! After having worked on this thing at the low-level detailed level for over half a year, this is by far the hardest part for me to see what someone just coming into this would need explained.

Edit: in a couple days I'm going to dig into the multiple bugs mentioned over the last few days, but I'd like to get at least one more solid day into the documentation. If I don't get a good start now, this could end up like a lot of stuff I do... lots of code, little documentation.

nlecaude · Feb 26, 2014

Anyone has a wav file they know works with audioPlaySDWav ? I tried several files here with no success and want to be sure the problem is not elsewhere... thanks !

saxen · Feb 26, 2014

Hi Paul,
I had a look at Audio Connections and Memory and Processor Usage and Interrupts, and it seems to be the most important part of the documentation. It took me some time to understand the general architecture of the library and the scheduling system. Having all this information in the documentation will definitely help. I would say this might be the getting started point! I'm also looking forward to read the part about "Optimizations with Cortex-M4 DSP Instructions".

I was thinking, wouldn't be better to create the connection by calling some method instead of creating an AudioConnection for each connection? I mean, something like connect(a,0,b,0), that may also support chains of connections, like connect(a,0,b,0,c,0). Maybe the method connect could just hide the AudioConnection declaration from the sketch and make things easier. What do you think?

Thanks,
Enrico

stephanschulz · Feb 27, 2014

will audio recording to the sd card be one of the things this library can ? i would love that

thanks for all your work.

PaulStoffregen · Feb 27, 2014

stephanschulz said:
will audio recording to the sd card be one of the things this library can ?

It's on my list of stuff to try, but right now my focus is writing the docs and fixing bugs in the objects that already exist.

Really, SD recording will depend on the performance of the SD card and Arduino's SD library, which is based on an older version of Bill's SdFat library.

rscott13 · Feb 27, 2014

FFT available blocking?

Question about the FFT library: is there anything (other than FIFO not ready) that would cause it to block? I'm observing the following behavior:

1) Using:

Code:

// Create Audio components
AudioInputI2Sslave	audioInput;
AudioAnalyzeFFT256      myFFT(20);
AudioAnalyzePrint       myprint("i2s");
AudioOutputI2Sslave	audioOutput;

// Create Audio connections between components
AudioConnection c1(audioInput, 0, audioOutput, 0);
AudioConnection c2(audioInput, 0, myFFT, 0);
AudioConnection c3(audioInput, 1, audioOutput, 1);
AudioConnection c4(audioInput, 0, myprint, 0);

2) Using AudioAnalyzePrint:

Code:

  Serial.println("Start sine test");
  musicPlayer.sineTest(100, 1);

  myprint.length(20);

  uint32_t count = 0;
  while (1) {
    Serial.print("Count: 0x"); Serial.println(count, HEX);
    count++;
    myprint.trigger();
    delay(1000);
  }

Results:
Count: 0x2
trigger i2s
-2549
-2481
-2393
-2286
-2160
-2016
-1857
-1682
-1493
-1292
-1081
-861
-635
-402
-167
70
307
540
769
992

3) Using FFT:

Code:

  Serial.println("Start sine test");
  musicPlayer.sineTest(100, 1);

  while (1) {
    if (myFFT.available()) {
      Serial.print("FFT: ");
      for (int i=0; i<12; i++) {
	Serial.print(myFFT.output[i]);
	Serial.print(",");
      }
      Serial.println();
    }
  }

results:

Start sine test
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,

So, the i2s FIFO is getting data, but the FFT is blocking. What's really interesting is if I change the sine test to an actual muisc file, the FFT magically unblocks:

Code:

while (1) {
    if (musicPlayer.stopped()) {
      Serial.println("Start playing music");
      musicPlayer.startPlayingFile("AND_YO~1.MP3");
      Serial.println("kickoff done");
    } else {
      if (myFFT.available()) {
	Serial.print("FFT: ");
 	for (int i=0; i<12; i++) {
	   Serial.print(myFFT.output[i]);
 	   Serial.print(",");
	 }
	 Serial.println();
      }
    }
  }

Results:
Start playing music
kickoff done
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,
FFT: 27,133,61,0,0,0,0,0,0,0,0,0,
FFT: 21,60,9,0,0,0,0,0,0,0,0,0,
FFT: 0,0,0,0,0,0,0,0,0,0,0,0,

The music clip starts silently. Here are some later results:

FFT: 330,239,183,234,96,77,42,134,99,63,9,36,
FFT: 329,239,271,319,159,118,176,206,111,36,43,58,
FFT: 378,305,265,232,91,99,165,204,9,81,27,0,
FFT: 389,314,160,152,63,88,160,198,39,9,0,0,
FFT: 401,343,114,165,102,82,171,229,78,30,9,0,
FFT: 331,294,203,201,118,78,124,121,0,0,0,0,
FFT: 241,202,174,189,95,58,96,164,70,45,9,0,
FFT: 217,231,210,199,94,58,119,81,9,0,0,0,

Background info: Using a VS1033 to decode MP3 files from an SD card. Modified Adafruit library used to play the MP3 files with VS1033 I2S output enabled. Eventual goal is to drive a strip of WS2811s with musical spectrum. (FFT may be overkill here - thinking about a 16 band filter bank instead...)

PaulStoffregen · Feb 27, 2014

rscott13 said:
So, the i2s FIFO is getting data, but the FFT is blocking.

First, let's talk about what you mean by "blocking".

Traditionally in software terminology, this word means code is waiting on data, thereby preventing other stuff from running. Is that what you're seeing, the rest of the audio stuff and/or your sketch not running?

There is a numerical resolution bug in the FFT code where small signals end up as all zeros. Unfortunately, it's inside the ARM math library, so this isn't something that's easy to fix. I'm going to work on it anyway, but first I'm working on the documentation and some of the other bugs.

rscott13 · Feb 27, 2014

PaulStoffregen said:
First, let's talk about what you mean by "blocking".

Traditionally in software terminology, this word means code is waiting on data, thereby preventing other stuff from running. Is that what you're seeing, the rest of the audio stuff and/or your sketch not running?

/QUOTE]

Exactly. I tried to show that the FFT sample stopped after a single line of output, when using the sine wave input, as opposed to when the music file was used as input. I looked at the FFT, and saw that it referenced some impressive sounding library call, and realized that my limit of software knowledge had been reached

el_supremo · Feb 27, 2014

If I read the code correctly, this:

Code:

AudioAnalyzeFFT256      myFFT(20);

tells the library to average 20 consecutive FFTs before returning the result. Reading 256 samples takes 5.8ms, so doing 20 buffers will take about 116ms (plus whatever time it takes to do the windowing and FFT).
You could try reducing the averaging to, say, 10 times and see how that affects your program.

@Paul:
I think you can speed up the copy_to_fft_buffer function. If you arrange that the destination buffer be on a 32-bit boundary and treat the samples as if they were unsigned, this should work. I've done a few quick tests and it seems to produce the same results as the original code. This evening I'll do some tests of whether it is any faster.

Code:

static void copy_to_fft_buffer(void *destination, const void *source)
{
	const uint16_t *src = (const uint16_t *)source;
	uint32_t *dst = (uint32_t *)destination;

	// TODO: optimize this
	for (int i=0; i < AUDIO_BLOCK_SAMPLES; i++) {
		*dst++ = *src++;  // real sample plus a zero for imaginary
	}
}

Pete

el_supremo · Feb 27, 2014

The output of AudioProcessorUsageMax was 17 in a test of the original code and is 15 using my version of copy_to_fft_buffer

Pete

PaulStoffregen · Feb 27, 2014

When I work on the FFT object, which might not be for a while, my first goal will be replacing the buggy ARM math library with actual FFT code. As long as it's under about 80% CPU, my first goal will be simply getting it working for all signal levels. I'm not planning to do any optimization work until the results are correct.

It's a lot easier to optimize already-debugged code than to debug already-optimized code!

Of course, I do have a pretty solid idea of how the Cortex-M4 DSP instructions work, so of course I'm not going to go down a path that's unworthy of later optimization. I'm just simply not going to worry about stuff like speeding up that copy operation which is specific to the ARM math library, when the math library code is due to be replaced with something that actually works.

Nantonos · Feb 28, 2014

PaulStoffregen said:
It's a lot easier to optimize already-debugged code than to debug already-optimized code!

Quoting for eminent requotability.

saxen · Feb 28, 2014

Hey Pete,
quick question, why do you use receiveWritable() in the fir filter object? When FIR_PASSTHRU, it does not seem to be needed. Otherwise, does arm_fir_q15 require it?

Thank you,
Enrico

Audio Library

Active member

Member

Well-known member

Active member

Well-known member

Active member

Well-known member

Active member

Well-known member

Well-known member

Well-known member

Well-known member

Well-known member

Well-known member

Active member

Well-known member

Well-known member

New member

Well-known member

New member

Well-known member

Well-known member

Well-known member

Well-known member

Active member