Accessing 128 bit sample packets

jjvande · Jan 19, 2020

Hi all,

I'm working on a university senior design team to implement an acoustic feedback cancellation device using the teensy4.0 and audio shield. At this point we'd like to use the CMSIS-DSP library, which has an easy to use normalized lms adaptive filter.

We've successfully got our teensy 4.0 modules up and running processing audio, but need to close the gap between the under-the-hood operations of the audio library, and the CMSIS-DSP library.

Our main challenge at this point is figuring out how to access the packets of data that stream between teensy audio objects. It's very easy to use the audio library with other audio library objects, but is there a way to store these packets to a buffer instead of sending straight to another processing or output object? If this is possible, we could "intercept" the packets with our normalized LMS filter, do some processing, and then send it to the audio output object.

The code below is just a basic sketch that passes audio between the line-in and line-out.

#include <Audio.h>
#include <Wire.h>
#include <SD.h>
#include <SPI.h>
#include <SerialFlash.h>

AudioInputI2S i2s_in;
AudioOutputI2S i2s_out;
AudioConnection patchcord1(i2s_in,0,i2s_out,0);
AudioConnection patchcord2(i2s_in,0,i2s_out,1);
AudioControlSGTL5000 sgtl5000;

void setup() {
Serial.begin(115200);
AudioMemory(10);
sgtl5000.enable();
sgtl5000.volume(0.5);
Serial.print("Setup Complete \n");
}

void loop() {
Serial.print("Streaming audio \n");
delay(1000);

}

el_supremo · Jan 19, 2020

Have a look at the play and record "queue" objects in the audio library.

Pete

el_supremo · Jan 19, 2020

Here's a simple use of the play and record queue objects. By default it plays stereo audio from line input to output (headphones). If a button is pushed, it simply swaps the left and right channels.
It was tested on a T3.6 and on a T4.

Pete

Code:

// Tested on a T3.6 using pin 32 for the button
// On T4 it uses pin 0
// Simple demo of the use of record and play queues
// If a button is pressed, switch the L and R channels
// of the audio. Otherwise, play straight through.
// By Pete El_Supremo 200119

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// Use this define to test that audio is playing
// Then undefine it to test the channel switching
//#define STRAIGHT_THRU

#ifdef STRAIGHT_THRU
AudioInputI2S            i2s_in;
AudioOutputI2S           i2s_out;
AudioConnection          patchCord1(i2s_in, 0, i2s_out, 0);
AudioConnection          patchCord2(i2s_in, 1, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;
#else
// GUItool: begin automatically generated code
AudioInputI2S            i2s_in;         //xy=357.8833312988281,290
AudioRecordQueue         Q_in_L;         //xy=533.8833312988281,215.88333129882812
AudioRecordQueue         Q_in_R;         //xy=538.8833312988281,341.8833312988281
AudioPlayQueue           Q_out_L;         //xy=681.8833312988281,217.88333129882812
AudioPlayQueue           Q_out_R;         //xy=682.8833312988281,338.8833312988281
AudioOutputI2S           i2s_out;        //xy=837.88330078125,276
AudioConnection          patchCord1(i2s_in, 0, Q_in_L, 0);
AudioConnection          patchCord2(i2s_in, 1, Q_in_R, 0);
AudioConnection          patchCord3(Q_out_L, 0, i2s_out, 0);
AudioConnection          patchCord4(Q_out_R, 0, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;       //xy=452.8833312988281,423
// GUItool: end automatically generated code
#endif

// Choose MICrophone or LINEIN
//const int myInput = AUDIO_INPUT_MIC;
const int myInput = AUDIO_INPUT_LINEIN;

short buf_L[AUDIO_BLOCK_SAMPLES];
short buf_R[AUDIO_BLOCK_SAMPLES];

// Button uses pin 32 on T3.6
//#define SWITCH_PIN 32
// Button uses pin 0 on T4
#define SWITCH_PIN 0

// debounce the button
#include <Bounce.h>
Bounce bouncer = Bounce( SWITCH_PIN,5 );

void setup(void)
{
  pinMode(SWITCH_PIN,INPUT_PULLUP);

  // Audio connections require memory. and the record queue
  // uses this memory to buffer incoming audio.
  AudioMemory(10);

  // Enable the audio shield. select input. and enable output
  sgtl5000.enable();
  sgtl5000.inputSelect(myInput);
  sgtl5000.volume(0.5);
#ifndef STRAIGHT_THRU
  // Start the record queues
  Q_in_L.begin();
  Q_in_R.begin();
#endif
}


void loop(void)
{
#ifndef STRAIGHT_THRU
  short *bp_L, *bp_R;

// Update the debouncer
  bouncer.update();

// When a left and right input buffer are available
// process them both
  if (Q_in_L.available() >= 1 && Q_in_R.available() >= 1) {
    bp_L = Q_in_L.readBuffer();
    bp_R = Q_in_R.readBuffer();
    // Copy the audio buffers to our memory
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      buf_L[i] = bp_L[i];
      buf_R[i] = bp_R[i];
    }
    // Free the input audio buffers
    Q_in_L.freeBuffer();
    Q_in_R.freeBuffer();

    // Process the buffers - if needed.
    // in this case - nothing to do

    if(bouncer.read()) {
      // Get pointers to "empty" output buffers
      bp_L = Q_out_L.getBuffer();
      bp_R = Q_out_R.getBuffer();
    } else {
      // Get pointers to "empty" output buffers
      // but switch them around
      bp_L = Q_out_R.getBuffer();
      bp_R = Q_out_L.getBuffer();
    }

    // copy the (un)processed data back to the output
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      bp_L[i] = buf_L[i];
      bp_R[i] = buf_R[i];
    }

    // and play them back into the audio queues
    Q_out_L.playBuffer();
    Q_out_R.playBuffer();
  }
#endif
}

jjvande · Jan 19, 2020

el_supremo said:

Here's a simple use of the play and record queue objects. By default it plays stereo audio from line input to output (headphones). If a button is pushed, it simply swaps the left and right channels.
It was tested on a T3.6 and on a T4.

Pete

Code:

// Tested on a T3.6 using pin 32 for the button
// On T4 it uses pin 0
// Simple demo of the use of record and play queues
// If a button is pressed, switch the L and R channels
// of the audio. Otherwise, play straight through.
// By Pete El_Supremo 200119

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// Use this define to test that audio is playing
// Then undefine it to test the channel switching
//#define STRAIGHT_THRU

#ifdef STRAIGHT_THRU
AudioInputI2S            i2s_in;
AudioOutputI2S           i2s_out;
AudioConnection          patchCord1(i2s_in, 0, i2s_out, 0);
AudioConnection          patchCord2(i2s_in, 1, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;
#else
// GUItool: begin automatically generated code
AudioInputI2S            i2s_in;         //xy=357.8833312988281,290
AudioRecordQueue         Q_in_L;         //xy=533.8833312988281,215.88333129882812
AudioRecordQueue         Q_in_R;         //xy=538.8833312988281,341.8833312988281
AudioPlayQueue           Q_out_L;         //xy=681.8833312988281,217.88333129882812
AudioPlayQueue           Q_out_R;         //xy=682.8833312988281,338.8833312988281
AudioOutputI2S           i2s_out;        //xy=837.88330078125,276
AudioConnection          patchCord1(i2s_in, 0, Q_in_L, 0);
AudioConnection          patchCord2(i2s_in, 1, Q_in_R, 0);
AudioConnection          patchCord3(Q_out_L, 0, i2s_out, 0);
AudioConnection          patchCord4(Q_out_R, 0, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;       //xy=452.8833312988281,423
// GUItool: end automatically generated code
#endif

// Choose MICrophone or LINEIN
//const int myInput = AUDIO_INPUT_MIC;
const int myInput = AUDIO_INPUT_LINEIN;

short buf_L[AUDIO_BLOCK_SAMPLES];
short buf_R[AUDIO_BLOCK_SAMPLES];

// Button uses pin 32 on T3.6
//#define SWITCH_PIN 32
// Button uses pin 0 on T4
#define SWITCH_PIN 0

// debounce the button
#include <Bounce.h>
Bounce bouncer = Bounce( SWITCH_PIN,5 );

void setup(void)
{
  pinMode(SWITCH_PIN,INPUT_PULLUP);

  // Audio connections require memory. and the record queue
  // uses this memory to buffer incoming audio.
  AudioMemory(10);

  // Enable the audio shield. select input. and enable output
  sgtl5000.enable();
  sgtl5000.inputSelect(myInput);
  sgtl5000.volume(0.5);
#ifndef STRAIGHT_THRU
  // Start the record queues
  Q_in_L.begin();
  Q_in_R.begin();
#endif
}


void loop(void)
{
#ifndef STRAIGHT_THRU
  short *bp_L, *bp_R;

// Update the debouncer
  bouncer.update();

// When a left and right input buffer are available
// process them both
  if (Q_in_L.available() >= 1 && Q_in_R.available() >= 1) {
    bp_L = Q_in_L.readBuffer();
    bp_R = Q_in_R.readBuffer();
    // Copy the audio buffers to our memory
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      buf_L[i] = bp_L[i];
      buf_R[i] = bp_R[i];
    }
    // Free the input audio buffers
    Q_in_L.freeBuffer();
    Q_in_R.freeBuffer();

    // Process the buffers - if needed.
    // in this case - nothing to do

    if(bouncer.read()) {
      // Get pointers to "empty" output buffers
      bp_L = Q_out_L.getBuffer();
      bp_R = Q_out_R.getBuffer();
    } else {
      // Get pointers to "empty" output buffers
      // but switch them around
      bp_L = Q_out_R.getBuffer();
      bp_R = Q_out_L.getBuffer();
    }

    // copy the (un)processed data back to the output
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      bp_L[i] = buf_L[i];
      bp_R[i] = buf_R[i];
    }

    // and play them back into the audio queues
    Q_out_L.playBuffer();
    Q_out_R.playBuffer();
  }
#endif
}

Pete,

I can't thank you enough! Being new to all this sometimes it's difficult to know where to begin looking. AudioRecordQueue and AudioPlayQueue will totally do the trick.

Thanks!

el_supremo · Jan 20, 2020

Being new to all this sometimes it's difficult to know where to begin looking

I know that feeling

Glad to be of help.

Pete

Diodac · Jan 29, 2023

el_supremo said:

Here's a simple use of the play and record queue objects. By default it plays stereo audio from line input to output (headphones). If a button is pushed, it simply swaps the left and right channels.
It was tested on a T3.6 and on a T4.

Pete

Code:

// Tested on a T3.6 using pin 32 for the button
// On T4 it uses pin 0
// Simple demo of the use of record and play queues
// If a button is pressed, switch the L and R channels
// of the audio. Otherwise, play straight through.
// By Pete El_Supremo 200119

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// Use this define to test that audio is playing
// Then undefine it to test the channel switching
//#define STRAIGHT_THRU

#ifdef STRAIGHT_THRU
AudioInputI2S            i2s_in;
AudioOutputI2S           i2s_out;
AudioConnection          patchCord1(i2s_in, 0, i2s_out, 0);
AudioConnection          patchCord2(i2s_in, 1, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;
#else
// GUItool: begin automatically generated code
AudioInputI2S            i2s_in;         //xy=357.8833312988281,290
AudioRecordQueue         Q_in_L;         //xy=533.8833312988281,215.88333129882812
AudioRecordQueue         Q_in_R;         //xy=538.8833312988281,341.8833312988281
AudioPlayQueue           Q_out_L;         //xy=681.8833312988281,217.88333129882812
AudioPlayQueue           Q_out_R;         //xy=682.8833312988281,338.8833312988281
AudioOutputI2S           i2s_out;        //xy=837.88330078125,276
AudioConnection          patchCord1(i2s_in, 0, Q_in_L, 0);
AudioConnection          patchCord2(i2s_in, 1, Q_in_R, 0);
AudioConnection          patchCord3(Q_out_L, 0, i2s_out, 0);
AudioConnection          patchCord4(Q_out_R, 0, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;       //xy=452.8833312988281,423
// GUItool: end automatically generated code
#endif

// Choose MICrophone or LINEIN
//const int myInput = AUDIO_INPUT_MIC;
const int myInput = AUDIO_INPUT_LINEIN;

short buf_L[AUDIO_BLOCK_SAMPLES];
short buf_R[AUDIO_BLOCK_SAMPLES];

// Button uses pin 32 on T3.6
//#define SWITCH_PIN 32
// Button uses pin 0 on T4
#define SWITCH_PIN 0

// debounce the button
#include <Bounce.h>
Bounce bouncer = Bounce( SWITCH_PIN,5 );

void setup(void)
{
  pinMode(SWITCH_PIN,INPUT_PULLUP);

  // Audio connections require memory. and the record queue
  // uses this memory to buffer incoming audio.
  AudioMemory(10);

  // Enable the audio shield. select input. and enable output
  sgtl5000.enable();
  sgtl5000.inputSelect(myInput);
  sgtl5000.volume(0.5);
#ifndef STRAIGHT_THRU
  // Start the record queues
  Q_in_L.begin();
  Q_in_R.begin();
#endif
}


void loop(void)
{
#ifndef STRAIGHT_THRU
  short *bp_L, *bp_R;

// Update the debouncer
  bouncer.update();

// When a left and right input buffer are available
// process them both
  if (Q_in_L.available() >= 1 && Q_in_R.available() >= 1) {
    bp_L = Q_in_L.readBuffer();
    bp_R = Q_in_R.readBuffer();
    // Copy the audio buffers to our memory
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      buf_L[i] = bp_L[i];
      buf_R[i] = bp_R[i];
    }
    // Free the input audio buffers
    Q_in_L.freeBuffer();
    Q_in_R.freeBuffer();

    // Process the buffers - if needed.
    // in this case - nothing to do

    if(bouncer.read()) {
      // Get pointers to "empty" output buffers
      bp_L = Q_out_L.getBuffer();
      bp_R = Q_out_R.getBuffer();
    } else {
      // Get pointers to "empty" output buffers
      // but switch them around
      bp_L = Q_out_R.getBuffer();
      bp_R = Q_out_L.getBuffer();
    }

    // copy the (un)processed data back to the output
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      bp_L[i] = buf_L[i];
      bp_R[i] = buf_R[i];
    }

    // and play them back into the audio queues
    Q_out_L.playBuffer();
    Q_out_R.playBuffer();
  }
#endif
}

Hi Pete
Actually I dealing with queue objects and have some problems. Maybe you could help me.
I try getting out audio from stream then processing it and after processing, send it back to audio stream and to the output. I see here your code and it give me hope

In my code something I do wrong because I have problem with artefacts noise which appears in interval of 6 minutes. Literally every 6 minutes coming strange noise.
I created nice multi effect with real time pitch shifting, flange, chorus and so on but noise problem makes my code useless

Here is my post about effects with my code
https://forum.pjrc.com/threads/7201...-flange-chorus-and-modulator-MULTI-FX-Project

Diodac · Jan 29, 2023

el_supremo said:

Here's a simple use of the play and record queue objects. By default it plays stereo audio from line input to output (headphones). If a button is pushed, it simply swaps the left and right channels.
It was tested on a T3.6 and on a T4.

Pete

Code:

// Tested on a T3.6 using pin 32 for the button
// On T4 it uses pin 0
// Simple demo of the use of record and play queues
// If a button is pressed, switch the L and R channels
// of the audio. Otherwise, play straight through.
// By Pete El_Supremo 200119

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// Use this define to test that audio is playing
// Then undefine it to test the channel switching
//#define STRAIGHT_THRU

#ifdef STRAIGHT_THRU
AudioInputI2S            i2s_in;
AudioOutputI2S           i2s_out;
AudioConnection          patchCord1(i2s_in, 0, i2s_out, 0);
AudioConnection          patchCord2(i2s_in, 1, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;
#else
// GUItool: begin automatically generated code
AudioInputI2S            i2s_in;         //xy=357.8833312988281,290
AudioRecordQueue         Q_in_L;         //xy=533.8833312988281,215.88333129882812
AudioRecordQueue         Q_in_R;         //xy=538.8833312988281,341.8833312988281
AudioPlayQueue           Q_out_L;         //xy=681.8833312988281,217.88333129882812
AudioPlayQueue           Q_out_R;         //xy=682.8833312988281,338.8833312988281
AudioOutputI2S           i2s_out;        //xy=837.88330078125,276
AudioConnection          patchCord1(i2s_in, 0, Q_in_L, 0);
AudioConnection          patchCord2(i2s_in, 1, Q_in_R, 0);
AudioConnection          patchCord3(Q_out_L, 0, i2s_out, 0);
AudioConnection          patchCord4(Q_out_R, 0, i2s_out, 1);
AudioControlSGTL5000     sgtl5000;       //xy=452.8833312988281,423
// GUItool: end automatically generated code
#endif

// Choose MICrophone or LINEIN
//const int myInput = AUDIO_INPUT_MIC;
const int myInput = AUDIO_INPUT_LINEIN;

short buf_L[AUDIO_BLOCK_SAMPLES];
short buf_R[AUDIO_BLOCK_SAMPLES];

// Button uses pin 32 on T3.6
//#define SWITCH_PIN 32
// Button uses pin 0 on T4
#define SWITCH_PIN 0

// debounce the button
#include <Bounce.h>
Bounce bouncer = Bounce( SWITCH_PIN,5 );

void setup(void)
{
  pinMode(SWITCH_PIN,INPUT_PULLUP);

  // Audio connections require memory. and the record queue
  // uses this memory to buffer incoming audio.
  AudioMemory(10);

  // Enable the audio shield. select input. and enable output
  sgtl5000.enable();
  sgtl5000.inputSelect(myInput);
  sgtl5000.volume(0.5);
#ifndef STRAIGHT_THRU
  // Start the record queues
  Q_in_L.begin();
  Q_in_R.begin();
#endif
}


void loop(void)
{
#ifndef STRAIGHT_THRU
  short *bp_L, *bp_R;

// Update the debouncer
  bouncer.update();

// When a left and right input buffer are available
// process them both
  if (Q_in_L.available() >= 1 && Q_in_R.available() >= 1) {
    bp_L = Q_in_L.readBuffer();
    bp_R = Q_in_R.readBuffer();
    // Copy the audio buffers to our memory
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      buf_L[i] = bp_L[i];
      buf_R[i] = bp_R[i];
    }
    // Free the input audio buffers
    Q_in_L.freeBuffer();
    Q_in_R.freeBuffer();

    // Process the buffers - if needed.
    // in this case - nothing to do

    if(bouncer.read()) {
      // Get pointers to "empty" output buffers
      bp_L = Q_out_L.getBuffer();
      bp_R = Q_out_R.getBuffer();
    } else {
      // Get pointers to "empty" output buffers
      // but switch them around
      bp_L = Q_out_R.getBuffer();
      bp_R = Q_out_L.getBuffer();
    }

    // copy the (un)processed data back to the output
    for(int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) {
      bp_L[i] = buf_L[i];
      bp_R[i] = buf_R[i];
    }

    // and play them back into the audio queues
    Q_out_L.playBuffer();
    Q_out_R.playBuffer();
  }
#endif
}

Regarding my previous post about problems with incoming noise. I fixed it thanks to view in your code

my problem comes from processing buffer outside queue.available function.
I rewrote my code and now all works perfect.
Later I share the code in my thread.
Thank you for sharing your example

it helped me better understand queue objects under audio library.

Accessing 128 bit sample packets

jjvande

Member

el_supremo

Well-known member

el_supremo

Well-known member

jjvande

Member

el_supremo

Well-known member

Diodac

Well-known member

Diodac

Well-known member