Audio Router Module?

[jkoffman]

I'm curious if there would be any sort of performance advantage if an audio router component existed. This would basically be like a mixer, but fixed input=output gain. I could see a use for a four in/four output crosspoint style. I am not versed enough in how the audio components are created, so this might not even be possible.

For those curious, my immediate use would be to take 4 inputs and be able to switch them around under software control - say someone plugs in two of the inputs backwards, we can fix that in software. I could also see a use when playing with various other effects and synths of being able to switch them in and out of the chain.

Thoughts?

[PaulStoffregen]

The mixer and amp features can do this now. When you set the gain to 0 or 1.0, they act like switches. Internally the code handles gain of 0 and 1.0 as special cases, routing the signals rather than going through the work of multiplying every sample.

So from a technical performance perspective, there's really little to nothing to be gained by having another object that does a complex switch matrix. It would be the same as just building it from mixers or amps.

From a human usability perspective, things aren't nearly so black & white. Technically, even the amp isn't needed, as it's just the same as using 1 channel of a mixer. But there was consistent feedback from people that using a 4 channel mixer to just switch or amplify or attenuate a single signal didn't feel right. So the 1 input, 1 output amp object was added to the library, and I spent a little time to create a nice graphic showing the 3 usage modes, since it was more about human convenience than a technical need.

Whether a switch matrix or other routing-oriented object makes sense from a usability perspective is a good question. The idea has come up a couple times. But the feedback so far is nothing like the steady chorus of requests for a simple switch or gain block when we only had the mixer, so at this point my main thinking is along the Zen of GitGub "Anything added dilutes everything else".

[jkoffman]

Hi Paul,

Ah, that makes sense. I noted that the amp mentions the 0 and 1.0 special cases, but the mixer didn't, so I wasn't sure. Thank you for the clarification!

My plan is to create this as a mixer. While the component would make things easier, I don't see it as much of a priority as what I'm sure your list is already jammed with.

Personally I'd vote for more work on DAC and ADC directly on the Teensy 4.0 over a router object!

Thank you!

[jkoffman]

Thinking about this further - any reason I couldn't make a custom mixer object of arbitrary size? I had a look at the mixer source code and it seems straightforward. So I wanted to make sure there wasn't some technical reason the existing object is 4 channels. I understand it might be a case of "this fits for most people and isn't crazy".

[jkoffman]

I'd like to try my hand at making a new audio block. I'd like to try to make a mixer object that has two outputs. I know that I could do this using two separate mixers as they currently are. I'd still like to try to see if this is something I can do though. I have not done a lot in C++, so some of this is a bit strange to me.

Before I start though, I saw this line on the page about creating new audio objects: "Only 1 block may be received from each channel during each run of update(), so there is no need to attempt receiving from any channel more than once. ". Now, this is under the receiveReadOnly() command, and I'd be using receiveWritable(). But, I wanted to check, can I read the same block of audio data twice? Or maybe I should I save to an intermediate variable so there's only one block access and then use the intermediate to do the writes to the two outputs.

Thoughts?

[wcalvert]

Audio is processed in blocks of 128 samples, so with a sample rate of 44.1kHz that means your code will be called once per 2.9 milliseconds. Your code needs to have only one occurrence of "audio_block_t *block = receiveWritable(channelNum);" per call to the update() method. You can receive multiple input channels like so:

for (channel=0; channel < 4; channel++) {
    out = receiveWritable(channel);
}

But you can't do this:

for (channel=0; channel < 4; channel++) {
    out = receiveWritable(0);
}

See the difference?

[jkoffman]

I see the difference but don't understand why I _can't_ do that. I understand why it wouldn't work, I'm basically reading the same channel over and over (receiveWritable(0)). So this wouldn't work, but would it throw an error and not compile?

[wcalvert]

It's a logical error, not a compile time error.

Think of it like this: when you call receiveWritable, you are taking possession of a piece of memory that is 128 samples long and 16 bits wide. Once that memory is in your possession, there's no more available until the next time update() is called.

[jkoffman]

Ok, so we're on the same page there. But if I called it again would it return the same block? Then I could make a second output channel by basically creating a second copy of the routine and running the routine again but pointing to a different output block.

My other thought would be to read one, store to a temp block, then use that as the source for the two passes for output channels.

[wcalvert]

My other thought would be to read one, store to a temp block, then use that as the source for the two passes for output channels.

Welllll... kinda. It might look something like this (pseudocode):

gain = .2;

out1 = allocate()
out2 = allocate()

for(int i=0; i<4; i++) {
    in = receiveWritable(i)
    for(int j=0; j<NUM_AUDIO_SAMPLES; j++) {
        out1->data[j] += in->data[j] * gain;
        out2->data[j] = out1->data[j];
    }
}

That is ignoring, of course, Paul's very clever optimizations that the Audio library is full of.

Edit, and thinking about it, this probably will not work exactly as I've written it, because when you call receiveWritable() the memory hasn't been blanked. You would need to blank it before starting to add stuff to it as I have shown it here.

[jkoffman]

I think that might be what is tripping me up...I'm not as clever as those optimizations!

What's confusing me the most at the moment is this:

	for (channel=0; channel < 4; channel++) {
		if (!out) {
			out = receiveWritable(channel);
			if (out) {
				int32_t mult = multiplier[channel];
				if (mult != MULTI_UNITYGAIN) applyGain(out->data, mult);
			}
		} else <snip>

The start of the loop makes sense. But the "if (!out)" doesn't fully. I am guessing we're doing an optimization such that if the out block is null (all zeros) then we don't need to _add_ the current data to it, we can just copy it over.

But...we check if the out block is null before reading it. Unless that's an autoincrementing pointer?

Am I even on the right track here?

[wcalvert]

The first time through the loop iteration, the output block hasn't been allocated yet. It checks for the allocation of the output block with the (!out) condition, allocates the output block using receiveWritable() and then applies gain to the input. The subsequent loop iterations will check for other input channels, apply gain and then add them to the output block.

[jkoffman]

Ok. So I guess maybe what I'm misunderstanding is the difference between receiveWritable() and receiveReadOnly(). It seems the basic difference is whether or not we're modifying the incoming data stream. For reference, here's the complete update routine:

void AudioMixer4::update(void)
{
	audio_block_t *in, *out=NULL;
	unsigned int channel;

	for (channel=0; channel < 4; channel++) {
		if (!out) {
			out = receiveWritable(channel);
			if (out) {
				int32_t mult = multiplier[channel];
				if (mult != MULTI_UNITYGAIN) applyGain(out->data, mult);
			}
		} else {
			in = receiveReadOnly(channel);
			if (in) {
				applyGainThenAdd(out->data, in->data, multiplier[channel]);
				release(in);
			}
		}
	}
	if (out) {
		transmit(out);
		release(out);
	}
}

The way I'm reading this, if the desired gain is unity, or the block is at 0 we use receiveReadOnly(), but if the output block is null (first pass) we use receiveWritable(). I would have thought we should have been able to use receiveReadOnly() for all of them since no matter whether this is the first pass through or not, we always take the data and jam it into a new buffer, not modify the input buffer. Where have I made a mistake in my beliefs?

[wcalvert]

It's more memory efficient to receive the first input block as writable so you can reuse it as the output block. That way you don't need to allocate an additional output block.

[jkoffman]

Ah, so is this why the first part of the if (!out) doesn't need a release(), but the else has a release(in) to free up the block since it uses receiveReadOnly(), correct?

In my case, since I'm reading the block and using it twice I feel I should be using receiveReadOnly() into a temp, then releasing it once I've updated the two output blocks.

However, the pseudocode you helpfully posted earlier you used receiveWritable(). You also updated the post (and I just noticed, oops) saying that it probably wouldn't work. Just for clarity, the update of your second output block is just a direct copy of the first, which is why the two lines look different, correct?

I'm going to try my hand at writing something and will post that to see how far off I am at understanding this.

Thank you for sticking with me!

[jkoffman]

Alright, here's what I came up with:

void AudioMixer4x2::update(void)
{
	audio_block_t *in, *intemp=NULL, *out1=NULL, *out2=NULL;
	unsigned int channel;

	for (channel=0; channel < 4; channel++) {
		in = receiveReadOnly(channel);
		if (in) {
			applyGainThenAdd(out1->data, in->data, multiplier1[channel]);
			applyGainThenAdd(out2->data, in->data, multiplier2[channel]);
			release(in);
			}		
	}
	if (out1) {
		transmit(out1);
		release(out1);
	}
	if (out2) {
		transmit(out2);
		release(out2);
	}

}

Some thoughts:
1. I don't allocate memory explicitly using allocate() the way you indicated.
2. I don't loop as long as there are samples the way you proposed. I didn't see in Paul's library that he did this, I am guessing this is an optimization on his part?
3. I may have an issue where I only release the out blocks if there is signal in that block. If it's null I don't release (or transmit), but I may grab them earlier. Not sure if this is an issue, just seems weird to me and possible that I made a mistake here.

Thoughts?

Edit: Just realized I have an extra struct in there. Just ignore intemp for now.

[WMXZ]

The first time around out1 and out2 are NULL, so code applyGainThenAdd could crash.
As out1 and out2 are local, program will never work

[jkoffman]

Thank you for the feedback. I've redone it, more closely modelled on the library mixer:

void AudioMixer4x2::update(void)
{
	audio_block_t *in, *intemp=NULL, *out1=NULL, *out2=NULL;
	unsigned int channel;
	int32_t mult;

	for (channel=0; channel < 4; channel++) {
		in = receiveReadOnly(channel);
		
		if (!out1) {
			out1 = in;
			if (out1) {
				mult = multiplier1[channel];
				if (mult != MULTI_UNITYGAIN) applyGain(out1->data, mult);
			}
			
		} else {
			if (in){
				applyGainThenAdd(out1->data, in->data, multiplier1[channel]);
			}
		}

		if (!out2) {
			out2 = in;
			if (out2) {
				mult = multiplier2[channel];
				if (mult != MULTI_UNITYGAIN) applyGain(out2->data, mult);
			}
			
		} else {
			if (in){
				applyGainThenAdd(out2->data, in->data, multiplier2[channel]);
			}
		}		
		release(in);
	}

	if (out1) {
		transmit(out1);
		release(out1);
	}
	if (out2) {
		transmit(out2);
		release(out2);
	}
}

I am not sure how to fix the out1 and out2 scope. Again, my C++ knowledge is not stellar, but I get that I'm defining them in the function so their scope will be within that function. However this is how Paul does it in mixer.cpp (https://github.com/PaulStoffregen/Au...ster/mixer.cpp) so I'm not sure how it works for his library.

Thank you again!

[jkoffman]

Hi folks,

With an overabundance of sudden free time, I'm now working on this again. For those catching up I have started with the generic 4 in, 1 out mixer and tried to add a second channel to it.

Here's the my current definition in the .h file:

class AudioMixer4x2 : public AudioStream
{
#if defined(__ARM_ARCH_7EM__)
public:
	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
		for (int i=0; i<4; i++) {
			multiplier0[i] = 65536;
			multiplier1[i] = 65536;
		}
	}
	virtual void update(void);
	void gain(unsigned int output, unsigned int channel, float gain) {
		if ((output < 0)||(output > 1)) return;
		if (channel >= 4) return;
		if (gain > 32767.0f) gain = 32767.0f;
		else if (gain < -32767.0f) gain = -32767.0f;
		if (output == 0) multiplier0[channel] = gain * 65536.0f; // TODO: proper roundoff?
		if (output == 1) multiplier1[channel] = gain * 65536.0f; // TODO: proper roundoff?
	}
private:
	int32_t multiplier0[4];
	int32_t multiplier1[4];
	audio_block_t *inputQueueArray[4];

#elif defined(KINETISL)
public:
	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
		for (int i=0; i<4; i++) {
			multiplier0[i] = 256;
			multiplier1[i] = 256;
		}
	}
	virtual void update(void);
	void gain(unsigned int output, unsigned int channel, float gain) {
		if (output < 0)||(output > 1) return;
		if (channel >= 4) return;
		if (gain > 127.0f) gain = 127.0f;
		else if (gain < -127.0f) gain = -127.0f;
		if (output == 0) multiplier0[channel] = gain * 256.0f; // TODO: proper roundoff?
		if (output == 1) multiplier1[channel] = gain * 256.0f; // TODO: proper roundoff?
	}
private:
	int32_t multiplier0[4];
	int32_t multiplier1[4];
	audio_block_t *inputQueueArray[4];
#endif
};

And here's the update function from .cpp:

void AudioMixer4x2::update(void)
{
//	audio_block_t *in, *intemp=NULL, *out1=NULL, *out2=NULL;
	audio_block_t *in, *out0=NULL, *out1=NULL;
	unsigned int channel;
	int32_t mult;

	for (channel=0; channel < 4; channel++) {
		in = receiveReadOnly(channel);
		
		if (!out0) {
			out0 = in;
			if (out0) {
				mult = multiplier0[channel];
				if (mult != MULTI_UNITYGAIN) applyGain(out0->data, mult);
			}
			
		} else {
			if (in){
				applyGainThenAdd(out0->data, in->data, multiplier0[channel]);
			}
		}

		if (!out1) {
			out1 = in;
			if (out1) {
				mult = multiplier1[channel];
				if (mult != MULTI_UNITYGAIN) applyGain(out1->data, mult);
			}
			
		} else {
			if (in){
				applyGainThenAdd(out1->data, in->data, multiplier1[channel]);
			}
		}
		
		release(in);
	
	}

	if (out0) {
		transmit(out0);
		release(out0);
	}
	if (out1) {
		transmit(out1);
		release(out1);
	}

}

Does it work? No! But...sort of.

Right now my test is just a simple USB in object and the output goes through I2S to the headphone jack on an audio shield. I have a .WAV file I'm playing back of me saying "left" on the left channel, and "right" on the right channel. If I connect USB in directly to I2S out, it works fine. If I connect through two regular mixer objects, it also works fine.

However, if I connect through my fancy new things are a bit more complicated. For the moment I have not tried adjusting any levels, just in case my gain() function is broken. When I play back my audio file, I do hear it, so that's a start. But the audio has crackles in it. Not once in a while, fairly constantly. Almost like clipping. Both left and right are heard out of output 0, and output 1 doesn't have any audio coming out. Level-wise it's about the same as without the mixer object, so I don't think I've accidentally cranked things up enough to clip.

I'm going to keep plugging away at it, but if anyone has any suggestions on what I've messed up I'm all ears!

[jkoffman]

I have reduced the code complexity to try to deal with the crackling issue. I've basically removed the second channel, and hard coded the amplification level to unity gain. Still the same crackling. I'm guessing it has to do with how I'm reading/writing the stream?

I'm hoping that fixing this will give some insight into why the second channel was totally dead. One thing at a time though!

Hmm..

void AudioMixer4x2::update(void)
{
	audio_block_t *in, *out0=NULL;
	unsigned int channel;
	int32_t mult;

	for (channel=0; channel < 4; channel++) {
		in = receiveReadOnly(channel);
		
		if (!out0) {
			out0 = in;
			if (out0) {
//				mult = multiplier0[channel];
				mult = MULTI_UNITYGAIN;
				if (mult != MULTI_UNITYGAIN) applyGain(out0->data, mult);
			}
			
		} else {
			if (in){
//				applyGainThenAdd(out0->data, in->data, multiplier0[channel]);
				applyGainThenAdd(out0->data, in->data, MULTI_UNITYGAIN);
			}
		}

		
		release(in);
	
	}

	if (out0) {
		transmit(out0);
		release(out0);
	}

}

[jkoffman]

Hi folks,

Well, it's been a couple of days, and I'm still banging my head against the wall. I did a deep dive on the forum archives and did find a few things that helped (ie I can now put sound out the second output if I want), but I'm still dealing with the crackling, even when I go back down to a 4 in 1 out mixer. Thinking about what I'm hearing I wonder if I am missing samples. The crackling doesn't seem to be clipping as the relative volume is not increasing. I am still playing my test file via USB which only has audio on one channel at a time. It also plays fine when I go through two 4x1 mixers, so I'm sure it's my mixer block that's the issue.

I added some serial prints so I can see if I'm accessing all parts of the routine. It looks like for each channel I am accessing the update() loops as I would expect. For the first time through the loop out0 is undefined, so we enter the outer part of the top loop. If there is input then we go through the inner part and add that value to the output. On subsequent passes through the channel for() loop, out0 is already defined so we go to the bottom outer. If there's incoming audio, we go to the bottom inner and add the value to the output. In my mind this all makes sense (though I am sure I could optimize things further, I know there is some redundancy the way the code is currently written), and it does work, but...crackles.

Any thoughts?

I am currently attempting to get things to work as a 4x1 mixer, then expand later to more outputs. I don't know if the clicking is being caused by how I am reading the input buffer, adding things together, or transmitting things on the output. Definitely feeling a bit in the weeds here and could use a bit of a steer. Thank you!

Relevant code here:
.h

class AudioMixer4x2 : public AudioStream
{
#if defined(__ARM_ARCH_7EM__)
public:
	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
		for (int i=0; i<4; i++) {
			multiplier0[i] = 65536;
			multiplier1[i] = 65536;
		}
	}
	virtual void update(void);
	void gain(unsigned int output, unsigned int channel, float gain) {
		if ((output < 0)||(output > 1)) return;
		if (channel >= 4) return;
		if (gain > 32767.0f) gain = 32767.0f;
		else if (gain < -32767.0f) gain = -32767.0f;
		if (output == 0) multiplier0[channel] = gain * 65536.0f; // TODO: proper roundoff?
		if (output == 1) multiplier1[channel] = gain * 65536.0f; // TODO: proper roundoff?
	}
private:
	int32_t multiplier0[4];
	int32_t multiplier1[4];
	audio_block_t *inputQueueArray[4];

#elif defined(KINETISL)
public:
	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
		for (int i=0; i<4; i++) {
			multiplier0[i] = 256;
			multiplier1[i] = 256;
		}
	}
	virtual void update(void);
	void gain(unsigned int output, unsigned int channel, float gain) {
		if (output < 0)||(output > 1) return;
		if (channel >= 4) return;
		if (gain > 127.0f) gain = 127.0f;
		else if (gain < -127.0f) gain = -127.0f;
		if (output == 0) multiplier0[channel] = gain * 256.0f; // TODO: proper roundoff?
		if (output == 1) multiplier1[channel] = gain * 256.0f; // TODO: proper roundoff?
	}
private:
	int32_t multiplier0[4];
	int32_t multiplier1[4];
	audio_block_t *inputQueueArray[4];
#endif
};

update() in .cpp

void AudioMixer4x2::update(void)
{

	audio_block_t *in, *out0=NULL;
	unsigned int channel;
	int32_t mult;

	Serial.printf("\nStart Update\n");

	for (channel=0; channel < 4; channel++) {
		in = receiveReadOnly(channel);

               Serial.printf("Channel %d \n", channel);


		if (!out0) {
			out0 = in;
			
			Serial.printf("Top Outer\n");
			
			if (out0) {

  			        Serial.printf("Top Inner\n");

				applyGainThenAdd(out0->data, in->data, MULTI_UNITYGAIN); // Hard coded to unity gain for testing
			}
			
		} else {

			Serial.printf("Bottom Outer\n");

			if (in){

			        Serial.printf("Bottom Inner\n");

				applyGainThenAdd(out0->data, in->data, MULTI_UNITYGAIN);  // Hard coded to unity gain for testing
			}
		}
		
		release(in);
	
	}

	if (out0) {
		transmit(out0);
		release(out0);
	}

}

[manicksan]

I have a untested solution that is based upon AudioSwitch_F32.cpp from Chip Audette, OpenAudio library

it's basically a Crosspoint switch (not mixer)

AudioCrosspointSwitch.tpp

/*
 * Created orginally: Chip Audette, OpenAudio, April 2019 AudioSwitch_F32.cpp
 * MIT License.  use at your own risk.
 * modified by: Jannik LF Svensson
 *              to a NI NO template based crosspoint switch
*/

template <int NI, int NO> void AudioCrosspointSwitch::update(void) {
    for (int i = 0; i < NO; i++) {
    
        audio_block_t *out=NULL;
        
        int inIndex = outputChannel[i];
        if (inIndex < 0) continue;
        
        out = receiveReadOnly(inIndex);
        
        if (!out) continue;

        AudioStream::transmit(out,i);
        AudioStream::release(out);
   }
}

AudioCrosspointSwitch.h

/*
 * AudioCrosspointSwitch.h
 * 
 * AudioSwitch<NI,NO>
 * Created orginal: Chip Audette, OpenAudio, April 2019
 * Purpose: Switch one input to any NO outputs, which should only trigger one of the NO
 *      audio processing paths (therebys saving computation on paths that you don't
 *      care about).        
 * This processes a single stream of audio data (ie, it is mono)       
 *          
 * MIT License.  use at your own risk.
*/

#ifndef AUDIOCROSSPOINTSWITCH_H
#define AUDIOCROSSPOINTSWITCH_H

#include <AudioStream.h>

template <int NI,int NO> class AudioCrosspointSwitch : public AudioStream {

public:
    AudioCrosspointSwitch() : AudioStream(NI, inputQueueArray) {
        for (int i=0; i<NO; i++) outputChannel[i] = -1; // -1 means unconnected
    }
    
    virtual void update(void);

    void setOutChannel(unsigned int out, unsigned int in) {
      if (out >= NO || out < 0) return;  //invalid!  stick with previous channel
      if (in >= NI) return;
      
      outputChannel[out] = in;
    }

  private:
    audio_block_t *inputQueueArray[NI];
    int outputChannel[NO];
};

#include "AudioCrosspointSwitch.tpp"

#endif

will this work?

[jkoffman]

Ah, interesting! I'm not sure I'd know exactly how to work this into the Teensy audio lib.

Unfortunately I think the larger issue for my application is that I believe it's 1 in, multiple out, correct?

What I'm looking to do is swap left/right channels. For that to work with this module I'd need two (one for each input channel) with two outputs each. So far no issue. But then I'd have to combine out 1 of each block together and out 2 of each block. Which would require mixers, so at that point I might as well accomplish the swapping with mixers.

If there were a way to make a 2 in/2 out (or some other arbitrary multi in/multi out) that would be very cool. Might be a bit beyond my capability, but who knows! One of the areas that I fell down in originally was just simply trying to add a second output onto the mixer object. I wasn't trying to control it separately, just wanted to pass the same audio to each. I failed, but if your code works then perhaps I can learn from it. This type of data handling isn't my specialty, but I will have a look next time I have some free time!

Thank you for sharing!

[manicksan]

there is this one
https://github.com/macaba/Audio/blob...sspoint_16.cpp
https://github.com/macaba/Audio/blob...rosspoint_16.h

from macaba
https://forum.pjrc.com/threads/43752-Crosspoint-Mixer

first "described" here
https://forum.pjrc.com/threads/41371...l=1#post142088

just have to change it into a c++ template
so that it become a arbitrary input/output

[manicksan]

If there were a way to make a 2 in/2 out (or some other arbitrary multi in/multi out) that would be very cool

That is exactly how c++ template works

the AudioCrosspointSwitch

is initialized with
AudioCrosspointSwitch<4,8> crossSwitch;

which is a 4 input 8 output AudioCrosspointSwitch

note.
I'm working with functionality that makes it possible to import whole custom libraries into the modified Audio Design Tool
that will make it possible to easily use custom libraries.