Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 21 of 21

Thread: Audio Router Module?

  1. #1

    Audio Router Module?

    I'm curious if there would be any sort of performance advantage if an audio router component existed. This would basically be like a mixer, but fixed input=output gain. I could see a use for a four in/four output crosspoint style. I am not versed enough in how the audio components are created, so this might not even be possible.

    For those curious, my immediate use would be to take 4 inputs and be able to switch them around under software control - say someone plugs in two of the inputs backwards, we can fix that in software. I could also see a use when playing with various other effects and synths of being able to switch them in and out of the chain.

    Thoughts?

  2. #2
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    21,498
    The mixer and amp features can do this now. When you set the gain to 0 or 1.0, they act like switches. Internally the code handles gain of 0 and 1.0 as special cases, routing the signals rather than going through the work of multiplying every sample.

    So from a technical performance perspective, there's really little to nothing to be gained by having another object that does a complex switch matrix. It would be the same as just building it from mixers or amps.

    From a human usability perspective, things aren't nearly so black & white. Technically, even the amp isn't needed, as it's just the same as using 1 channel of a mixer. But there was consistent feedback from people that using a 4 channel mixer to just switch or amplify or attenuate a single signal didn't feel right. So the 1 input, 1 output amp object was added to the library, and I spent a little time to create a nice graphic showing the 3 usage modes, since it was more about human convenience than a technical need.

    Whether a switch matrix or other routing-oriented object makes sense from a usability perspective is a good question. The idea has come up a couple times. But the feedback so far is nothing like the steady chorus of requests for a simple switch or gain block when we only had the mixer, so at this point my main thinking is along the Zen of GitGub "Anything added dilutes everything else".

  3. #3
    Hi Paul,

    Ah, that makes sense. I noted that the amp mentions the 0 and 1.0 special cases, but the mixer didn't, so I wasn't sure. Thank you for the clarification!

    My plan is to create this as a mixer. While the component would make things easier, I don't see it as much of a priority as what I'm sure your list is already jammed with.

    Personally I'd vote for more work on DAC and ADC directly on the Teensy 4.0 over a router object!

    Thank you!

  4. #4
    Thinking about this further - any reason I couldn't make a custom mixer object of arbitrary size? I had a look at the mixer source code and it seems straightforward. So I wanted to make sure there wasn't some technical reason the existing object is 4 channels. I understand it might be a case of "this fits for most people and isn't crazy".

  5. #5
    I'd like to try my hand at making a new audio block. I'd like to try to make a mixer object that has two outputs. I know that I could do this using two separate mixers as they currently are. I'd still like to try to see if this is something I can do though. I have not done a lot in C++, so some of this is a bit strange to me.

    Before I start though, I saw this line on the page about creating new audio objects: "Only 1 block may be received from each channel during each run of update(), so there is no need to attempt receiving from any channel more than once. ". Now, this is under the receiveReadOnly() command, and I'd be using receiveWritable(). But, I wanted to check, can I read the same block of audio data twice? Or maybe I should I save to an intermediate variable so there's only one block access and then use the intermediate to do the writes to the two outputs.

    Thoughts?

  6. #6
    Senior Member
    Join Date
    Feb 2015
    Posts
    213
    Audio is processed in blocks of 128 samples, so with a sample rate of 44.1kHz that means your code will be called once per 2.9 milliseconds. Your code needs to have only one occurrence of "audio_block_t *block = receiveWritable(channelNum);" per call to the update() method. You can receive multiple input channels like so:

    Code:
    for (channel=0; channel < 4; channel++) {
        out = receiveWritable(channel);
    }
    But you can't do this:
    Code:
    for (channel=0; channel < 4; channel++) {
        out = receiveWritable(0);
    }
    See the difference?

  7. #7
    I see the difference but don't understand why I _can't_ do that. I understand why it wouldn't work, I'm basically reading the same channel over and over (receiveWritable(0)). So this wouldn't work, but would it throw an error and not compile?

  8. #8
    Senior Member
    Join Date
    Feb 2015
    Posts
    213
    It's a logical error, not a compile time error.

    Think of it like this: when you call receiveWritable, you are taking possession of a piece of memory that is 128 samples long and 16 bits wide. Once that memory is in your possession, there's no more available until the next time update() is called.

  9. #9
    Ok, so we're on the same page there. But if I called it again would it return the same block? Then I could make a second output channel by basically creating a second copy of the routine and running the routine again but pointing to a different output block.

    My other thought would be to read one, store to a temp block, then use that as the source for the two passes for output channels.

  10. #10
    Senior Member
    Join Date
    Feb 2015
    Posts
    213
    Quote Originally Posted by jkoffman View Post
    My other thought would be to read one, store to a temp block, then use that as the source for the two passes for output channels.
    Welllll... kinda. It might look something like this (pseudocode):
    Code:
    gain = .2;
    
    out1 = allocate()
    out2 = allocate()
    
    for(int i=0; i<4; i++) {
        in = receiveWritable(i)
        for(int j=0; j<NUM_AUDIO_SAMPLES; j++) {
            out1->data[j] += in->data[j] * gain;
            out2->data[j] = out1->data[j];
        }
    }
    That is ignoring, of course, Paul's very clever optimizations that the Audio library is full of.

    Edit, and thinking about it, this probably will not work exactly as I've written it, because when you call receiveWritable() the memory hasn't been blanked. You would need to blank it before starting to add stuff to it as I have shown it here.

  11. #11
    I think that might be what is tripping me up...I'm not as clever as those optimizations!

    What's confusing me the most at the moment is this:
    Code:
    	for (channel=0; channel < 4; channel++) {
    		if (!out) {
    			out = receiveWritable(channel);
    			if (out) {
    				int32_t mult = multiplier[channel];
    				if (mult != MULTI_UNITYGAIN) applyGain(out->data, mult);
    			}
    		} else <snip>
    The start of the loop makes sense. But the "if (!out)" doesn't fully. I am guessing we're doing an optimization such that if the out block is null (all zeros) then we don't need to _add_ the current data to it, we can just copy it over.

    But...we check if the out block is null before reading it. Unless that's an autoincrementing pointer?

    Am I even on the right track here?

  12. #12
    Senior Member
    Join Date
    Feb 2015
    Posts
    213
    The first time through the loop iteration, the output block hasn't been allocated yet. It checks for the allocation of the output block with the (!out) condition, allocates the output block using receiveWritable() and then applies gain to the input. The subsequent loop iterations will check for other input channels, apply gain and then add them to the output block.

  13. #13
    Ok. So I guess maybe what I'm misunderstanding is the difference between receiveWritable() and receiveReadOnly(). It seems the basic difference is whether or not we're modifying the incoming data stream. For reference, here's the complete update routine:

    Code:
    void AudioMixer4::update(void)
    {
    	audio_block_t *in, *out=NULL;
    	unsigned int channel;
    
    	for (channel=0; channel < 4; channel++) {
    		if (!out) {
    			out = receiveWritable(channel);
    			if (out) {
    				int32_t mult = multiplier[channel];
    				if (mult != MULTI_UNITYGAIN) applyGain(out->data, mult);
    			}
    		} else {
    			in = receiveReadOnly(channel);
    			if (in) {
    				applyGainThenAdd(out->data, in->data, multiplier[channel]);
    				release(in);
    			}
    		}
    	}
    	if (out) {
    		transmit(out);
    		release(out);
    	}
    }
    The way I'm reading this, if the desired gain is unity, or the block is at 0 we use receiveReadOnly(), but if the output block is null (first pass) we use receiveWritable(). I would have thought we should have been able to use receiveReadOnly() for all of them since no matter whether this is the first pass through or not, we always take the data and jam it into a new buffer, not modify the input buffer. Where have I made a mistake in my beliefs?

  14. #14
    Senior Member
    Join Date
    Feb 2015
    Posts
    213
    It's more memory efficient to receive the first input block as writable so you can reuse it as the output block. That way you don't need to allocate an additional output block.

  15. #15
    Ah, so is this why the first part of the if (!out) doesn't need a release(), but the else has a release(in) to free up the block since it uses receiveReadOnly(), correct?

    In my case, since I'm reading the block and using it twice I feel I should be using receiveReadOnly() into a temp, then releasing it once I've updated the two output blocks.

    However, the pseudocode you helpfully posted earlier you used receiveWritable(). You also updated the post (and I just noticed, oops) saying that it probably wouldn't work. Just for clarity, the update of your second output block is just a direct copy of the first, which is why the two lines look different, correct?

    I'm going to try my hand at writing something and will post that to see how far off I am at understanding this.

    Thank you for sticking with me!

  16. #16
    Alright, here's what I came up with:

    Code:
    void AudioMixer4x2::update(void)
    {
    	audio_block_t *in, *intemp=NULL, *out1=NULL, *out2=NULL;
    	unsigned int channel;
    
    	for (channel=0; channel < 4; channel++) {
    		in = receiveReadOnly(channel);
    		if (in) {
    			applyGainThenAdd(out1->data, in->data, multiplier1[channel]);
    			applyGainThenAdd(out2->data, in->data, multiplier2[channel]);
    			release(in);
    			}		
    	}
    	if (out1) {
    		transmit(out1);
    		release(out1);
    	}
    	if (out2) {
    		transmit(out2);
    		release(out2);
    	}
    
    }
    Some thoughts:
    1. I don't allocate memory explicitly using allocate() the way you indicated.
    2. I don't loop as long as there are samples the way you proposed. I didn't see in Paul's library that he did this, I am guessing this is an optimization on his part?
    3. I may have an issue where I only release the out blocks if there is signal in that block. If it's null I don't release (or transmit), but I may grab them earlier. Not sure if this is an issue, just seems weird to me and possible that I made a mistake here.

    Thoughts?

    Edit: Just realized I have an extra struct in there. Just ignore intemp for now.

  17. #17
    Senior Member
    Join Date
    Jul 2014
    Posts
    2,570
    The first time around out1 and out2 are NULL, so code applyGainThenAdd could crash.
    As out1 and out2 are local, program will never work

  18. #18
    Thank you for the feedback. I've redone it, more closely modelled on the library mixer:

    Code:
    void AudioMixer4x2::update(void)
    {
    	audio_block_t *in, *intemp=NULL, *out1=NULL, *out2=NULL;
    	unsigned int channel;
    	int32_t mult;
    
    	for (channel=0; channel < 4; channel++) {
    		in = receiveReadOnly(channel);
    		
    		if (!out1) {
    			out1 = in;
    			if (out1) {
    				mult = multiplier1[channel];
    				if (mult != MULTI_UNITYGAIN) applyGain(out1->data, mult);
    			}
    			
    		} else {
    			if (in){
    				applyGainThenAdd(out1->data, in->data, multiplier1[channel]);
    			}
    		}
    
    		if (!out2) {
    			out2 = in;
    			if (out2) {
    				mult = multiplier2[channel];
    				if (mult != MULTI_UNITYGAIN) applyGain(out2->data, mult);
    			}
    			
    		} else {
    			if (in){
    				applyGainThenAdd(out2->data, in->data, multiplier2[channel]);
    			}
    		}		
    		release(in);
    	}
    
    	if (out1) {
    		transmit(out1);
    		release(out1);
    	}
    	if (out2) {
    		transmit(out2);
    		release(out2);
    	}
    }
    I am not sure how to fix the out1 and out2 scope. Again, my C++ knowledge is not stellar, but I get that I'm defining them in the function so their scope will be within that function. However this is how Paul does it in mixer.cpp (https://github.com/PaulStoffregen/Au...ster/mixer.cpp) so I'm not sure how it works for his library.

    Thank you again!

  19. #19
    Hi folks,

    With an overabundance of sudden free time, I'm now working on this again. For those catching up I have started with the generic 4 in, 1 out mixer and tried to add a second channel to it.

    Here's the my current definition in the .h file:

    Code:
    class AudioMixer4x2 : public AudioStream
    {
    #if defined(__ARM_ARCH_7EM__)
    public:
    	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
    		for (int i=0; i<4; i++) {
    			multiplier0[i] = 65536;
    			multiplier1[i] = 65536;
    		}
    	}
    	virtual void update(void);
    	void gain(unsigned int output, unsigned int channel, float gain) {
    		if ((output < 0)||(output > 1)) return;
    		if (channel >= 4) return;
    		if (gain > 32767.0f) gain = 32767.0f;
    		else if (gain < -32767.0f) gain = -32767.0f;
    		if (output == 0) multiplier0[channel] = gain * 65536.0f; // TODO: proper roundoff?
    		if (output == 1) multiplier1[channel] = gain * 65536.0f; // TODO: proper roundoff?
    	}
    private:
    	int32_t multiplier0[4];
    	int32_t multiplier1[4];
    	audio_block_t *inputQueueArray[4];
    
    #elif defined(KINETISL)
    public:
    	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
    		for (int i=0; i<4; i++) {
    			multiplier0[i] = 256;
    			multiplier1[i] = 256;
    		}
    	}
    	virtual void update(void);
    	void gain(unsigned int output, unsigned int channel, float gain) {
    		if (output < 0)||(output > 1) return;
    		if (channel >= 4) return;
    		if (gain > 127.0f) gain = 127.0f;
    		else if (gain < -127.0f) gain = -127.0f;
    		if (output == 0) multiplier0[channel] = gain * 256.0f; // TODO: proper roundoff?
    		if (output == 1) multiplier1[channel] = gain * 256.0f; // TODO: proper roundoff?
    	}
    private:
    	int32_t multiplier0[4];
    	int32_t multiplier1[4];
    	audio_block_t *inputQueueArray[4];
    #endif
    };
    And here's the update function from .cpp:

    Code:
    void AudioMixer4x2::update(void)
    {
    //	audio_block_t *in, *intemp=NULL, *out1=NULL, *out2=NULL;
    	audio_block_t *in, *out0=NULL, *out1=NULL;
    	unsigned int channel;
    	int32_t mult;
    
    	for (channel=0; channel < 4; channel++) {
    		in = receiveReadOnly(channel);
    		
    		if (!out0) {
    			out0 = in;
    			if (out0) {
    				mult = multiplier0[channel];
    				if (mult != MULTI_UNITYGAIN) applyGain(out0->data, mult);
    			}
    			
    		} else {
    			if (in){
    				applyGainThenAdd(out0->data, in->data, multiplier0[channel]);
    			}
    		}
    
    		if (!out1) {
    			out1 = in;
    			if (out1) {
    				mult = multiplier1[channel];
    				if (mult != MULTI_UNITYGAIN) applyGain(out1->data, mult);
    			}
    			
    		} else {
    			if (in){
    				applyGainThenAdd(out1->data, in->data, multiplier1[channel]);
    			}
    		}
    		
    		release(in);
    	
    	}
    
    	if (out0) {
    		transmit(out0);
    		release(out0);
    	}
    	if (out1) {
    		transmit(out1);
    		release(out1);
    	}
    
    }
    Does it work? No! But...sort of.

    Right now my test is just a simple USB in object and the output goes through I2S to the headphone jack on an audio shield. I have a .WAV file I'm playing back of me saying "left" on the left channel, and "right" on the right channel. If I connect USB in directly to I2S out, it works fine. If I connect through two regular mixer objects, it also works fine.

    However, if I connect through my fancy new things are a bit more complicated. For the moment I have not tried adjusting any levels, just in case my gain() function is broken. When I play back my audio file, I do hear it, so that's a start. But the audio has crackles in it. Not once in a while, fairly constantly. Almost like clipping. Both left and right are heard out of output 0, and output 1 doesn't have any audio coming out. Level-wise it's about the same as without the mixer object, so I don't think I've accidentally cranked things up enough to clip.

    I'm going to keep plugging away at it, but if anyone has any suggestions on what I've messed up I'm all ears!
    Last edited by jkoffman; 03-14-2020 at 02:48 PM.

  20. #20
    I have reduced the code complexity to try to deal with the crackling issue. I've basically removed the second channel, and hard coded the amplification level to unity gain. Still the same crackling. I'm guessing it has to do with how I'm reading/writing the stream?

    I'm hoping that fixing this will give some insight into why the second channel was totally dead. One thing at a time though!

    Hmm..

    Code:
    void AudioMixer4x2::update(void)
    {
    	audio_block_t *in, *out0=NULL;
    	unsigned int channel;
    	int32_t mult;
    
    	for (channel=0; channel < 4; channel++) {
    		in = receiveReadOnly(channel);
    		
    		if (!out0) {
    			out0 = in;
    			if (out0) {
    //				mult = multiplier0[channel];
    				mult = MULTI_UNITYGAIN;
    				if (mult != MULTI_UNITYGAIN) applyGain(out0->data, mult);
    			}
    			
    		} else {
    			if (in){
    //				applyGainThenAdd(out0->data, in->data, multiplier0[channel]);
    				applyGainThenAdd(out0->data, in->data, MULTI_UNITYGAIN);
    			}
    		}
    
    		
    		release(in);
    	
    	}
    
    	if (out0) {
    		transmit(out0);
    		release(out0);
    	}
    
    }

  21. #21
    Hi folks,

    Well, it's been a couple of days, and I'm still banging my head against the wall. I did a deep dive on the forum archives and did find a few things that helped (ie I can now put sound out the second output if I want), but I'm still dealing with the crackling, even when I go back down to a 4 in 1 out mixer. Thinking about what I'm hearing I wonder if I am missing samples. The crackling doesn't seem to be clipping as the relative volume is not increasing. I am still playing my test file via USB which only has audio on one channel at a time. It also plays fine when I go through two 4x1 mixers, so I'm sure it's my mixer block that's the issue.

    I added some serial prints so I can see if I'm accessing all parts of the routine. It looks like for each channel I am accessing the update() loops as I would expect. For the first time through the loop out0 is undefined, so we enter the outer part of the top loop. If there is input then we go through the inner part and add that value to the output. On subsequent passes through the channel for() loop, out0 is already defined so we go to the bottom outer. If there's incoming audio, we go to the bottom inner and add the value to the output. In my mind this all makes sense (though I am sure I could optimize things further, I know there is some redundancy the way the code is currently written), and it does work, but...crackles.

    Any thoughts?

    I am currently attempting to get things to work as a 4x1 mixer, then expand later to more outputs. I don't know if the clicking is being caused by how I am reading the input buffer, adding things together, or transmitting things on the output. Definitely feeling a bit in the weeds here and could use a bit of a steer. Thank you!

    Relevant code here:
    .h
    Code:
    class AudioMixer4x2 : public AudioStream
    {
    #if defined(__ARM_ARCH_7EM__)
    public:
    	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
    		for (int i=0; i<4; i++) {
    			multiplier0[i] = 65536;
    			multiplier1[i] = 65536;
    		}
    	}
    	virtual void update(void);
    	void gain(unsigned int output, unsigned int channel, float gain) {
    		if ((output < 0)||(output > 1)) return;
    		if (channel >= 4) return;
    		if (gain > 32767.0f) gain = 32767.0f;
    		else if (gain < -32767.0f) gain = -32767.0f;
    		if (output == 0) multiplier0[channel] = gain * 65536.0f; // TODO: proper roundoff?
    		if (output == 1) multiplier1[channel] = gain * 65536.0f; // TODO: proper roundoff?
    	}
    private:
    	int32_t multiplier0[4];
    	int32_t multiplier1[4];
    	audio_block_t *inputQueueArray[4];
    
    #elif defined(KINETISL)
    public:
    	AudioMixer4x2(void) : AudioStream(4, inputQueueArray) {
    		for (int i=0; i<4; i++) {
    			multiplier0[i] = 256;
    			multiplier1[i] = 256;
    		}
    	}
    	virtual void update(void);
    	void gain(unsigned int output, unsigned int channel, float gain) {
    		if (output < 0)||(output > 1) return;
    		if (channel >= 4) return;
    		if (gain > 127.0f) gain = 127.0f;
    		else if (gain < -127.0f) gain = -127.0f;
    		if (output == 0) multiplier0[channel] = gain * 256.0f; // TODO: proper roundoff?
    		if (output == 1) multiplier1[channel] = gain * 256.0f; // TODO: proper roundoff?
    	}
    private:
    	int32_t multiplier0[4];
    	int32_t multiplier1[4];
    	audio_block_t *inputQueueArray[4];
    #endif
    };
    update() in .cpp

    Code:
    void AudioMixer4x2::update(void)
    {
    
    	audio_block_t *in, *out0=NULL;
    	unsigned int channel;
    	int32_t mult;
    
    	Serial.printf("\nStart Update\n");
    
    	for (channel=0; channel < 4; channel++) {
    		in = receiveReadOnly(channel);
    
                   Serial.printf("Channel %d \n", channel);
    
    
    		if (!out0) {
    			out0 = in;
    			
    			Serial.printf("Top Outer\n");
    			
    			if (out0) {
    
      			        Serial.printf("Top Inner\n");
    
    				applyGainThenAdd(out0->data, in->data, MULTI_UNITYGAIN); // Hard coded to unity gain for testing
    			}
    			
    		} else {
    
    			Serial.printf("Bottom Outer\n");
    
    			if (in){
    
    			        Serial.printf("Bottom Inner\n");
    
    				applyGainThenAdd(out0->data, in->data, MULTI_UNITYGAIN);  // Hard coded to unity gain for testing
    			}
    		}
    		
    		release(in);
    	
    	}
    
    	if (out0) {
    		transmit(out0);
    		release(out0);
    	}
    
    }

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •