Understanding queue and buffers

Status
Not open for further replies.

biccius

Member
Hi, everybody

I'm starting to use a Teensy 4.0 with Audio Shield (SGTL5000) and Microphone and i'm interested in understanding how the library works and in particular the queues.

I'm trying to use the incoming and outgoing queue object to try to create a delay effect.

So my purpose is to reproduce correctly the input stream after a defined number of milliseconds.

I already know that an object that already allows this is implemented but I would like to try to get a "similar" effect by manipulating the buffers at a lower level for undestanding the functions of the Audio Library Objects.

This is the code i'm working on.

What it should do is delay playback by 500 ms but it doesn't seem to work.

In the headphones the sound is reproduced without a perceptible delay.

What am I doing wrong?
Are there any examples I can take my cue from?

Thanks

Code:
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>


AudioRecordQueue         queueRecord;        //xy=208,57
AudioPlayQueue           queuePlay;          //xy=327,55
AudioInputI2S            i2s1Record;         //xy=149,228
AudioAnalyzePeak         peak1;          	 //xy=407,266
//AudioAnalyzePeak         peak2;          	 //xy=407,266
AudioOutputI2S           i2s2Play;           //xy=447,200
AudioConnection          patchCord1(i2s1Record, 0, queueRecord, 0);
AudioConnection          patchCord2(queuePlay,  0, i2s2Play, 1);
AudioConnection          patchCord3(i2s1Record, 0, peak1, 0);
//AudioConnection          patchCord4(i2s1, 0, peak2, 1);
AudioControlSGTL5000     sgtl5000_1;     //xy=339,338


#define 		MAX_SAMPLES 	128
#define 		MAX_QUEUE_SIZE  1000

uint16_t 		buffer[MAX_SAMPLES * MAX_QUEUE_SIZE];
uint8_t  		record_index = 0;
uint8_t  		play_index = 0;
uint8_t  		record_offset = 0;
uint8_t  		play_offset = 0;
unsigned long 	millis_buffer[MAX_QUEUE_SIZE];


elapsedMillis mymillis;


// INPUT     OUTPUT
// i2s1 L >> i2s2 L
// i2s1 R >> i2s2 R

void setup()
{
  Serial.begin(115200);
  AudioMemory(100);
  sgtl5000_1.enable();
  sgtl5000_1.volume(0.5);
  sgtl5000_1.inputSelect(AUDIO_INPUT_MIC);
  sgtl5000_1.micGain(36);
  //sgtl5000_1.lineInLevel(0);
  sgtl5000_1.lineOutLevel(13);
  queueRecord.begin();
  mymillis = 0;
}

void printPeak()
{
	Serial.println(peak1.read());
	delay(5);
}



void loop()
{

	//printPeak();
	//delay(1000);

	if(queueRecord.available() >= 2)
	{   //input buffer

	    memcpy(buffer + record_offset, queueRecord.readBuffer(), MAX_SAMPLES);
	    queueRecord.freeBuffer();
	    millis_buffer[record_index] = mymillis;


	    record_offset+=128;
	    record_index+=1;

	    if (record_index >= MAX_QUEUE_SIZE)
	    {
	    	record_offset=0;
	    	record_index=0;
	    }

	}


	if (mymillis - millis_buffer[play_index] > 500)
	{

		memcpy(queuePlay.getBuffer(), buffer + play_offset, MAX_SAMPLES);
		queuePlay.playBuffer();
		millis_buffer[play_index] = 0;

		play_offset+=128;
		play_index+=1;

	    if (play_index >= MAX_QUEUE_SIZE)
		{
	    		play_index=0;
	    		play_offset=0;
		}

	}
}
 
Ok, previous code was wrong.
Now should be right:

Code:
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>


AudioRecordQueue         queueRecord;
AudioPlayQueue           queuePlay;
AudioInputI2S            i2s1Record;
AudioAnalyzePeak         peak1;
//AudioAnalyzePeak         peak2;
AudioOutputI2S           i2s2Play;
AudioConnection          patchCord1(i2s1Record, 0, queueRecord, 0);
AudioConnection          patchCord2(queuePlay,  0, i2s2Play, 1);
AudioConnection          patchCord3(i2s1Record, 0, peak1, 0);
//AudioConnection          patchCord4(i2s1, 0, peak2, 1);
AudioControlSGTL5000     sgtl5000_1;


#define 		MAX_SAMPLES 	128
#define 		MAX_QUEUE_SIZE  1536

uint16_t 		buffer[MAX_SAMPLES * MAX_QUEUE_SIZE];
int32_t			record_offset = 0;
int32_t  		play_offset = 0;


elapsedMillis 	mymillis;


// INPUT     OUTPUT
// i2s1 L >> i2s2 L
// i2s1 R >> i2s2 R

void setup()
{
  Serial.begin(115200);
  AudioMemory(256);
  sgtl5000_1.enable();
  sgtl5000_1.volume(0.5);
  sgtl5000_1.inputSelect(AUDIO_INPUT_MIC);
  sgtl5000_1.micGain(36);
  //sgtl5000_1.lineInLevel(0);
  sgtl5000_1.lineOutLevel(13);
  queueRecord.begin();
  mymillis = 0;
}

void printPeak()
{
	Serial.println(peak1.read());
	delay(5);
}


void i2s_to_buffer()
{
	memcpy(buffer + record_offset, queueRecord.readBuffer(), MAX_SAMPLES);
	queueRecord.freeBuffer();
	record_offset+=MAX_SAMPLES;
	if (record_offset >= (MAX_SAMPLES * MAX_QUEUE_SIZE))	record_offset=0;

}

void buffer_to_i2s()
{
   memcpy(queuePlay.getBuffer(), buffer + play_offset , MAX_SAMPLES);
    queuePlay.playBuffer();
    play_offset+=MAX_SAMPLES;
    if (play_offset >= (MAX_SAMPLES * MAX_QUEUE_SIZE)) play_offset=0;


}


void loop()
{

	//printPeak();
	//delay(1000);

	if(queueRecord.available() >= 2)
	{

		i2s_to_buffer(); // buffer samples


		if (mymillis > 500) //if more than 500ms elapsed from last buffer record
		{
			buffer_to_i2s(); //execute last buffered sample

		}

		//Serial.print(record_offset);
		//Serial.print(" ");
		//Serial.print(play_offset);
		//Serial.println();

	}// end if(queueRecord.available() >= 2)



}



But what i hear seems to be distorted.
If i don't put a delay (for example: mymillis > 0) the sound in the phones is the exact reproduction of what the mic record
Someone can explain what's wrong?

Code:
	if(queueRecord.available() >= 2)
	{

		i2s_to_buffer(); // buffer samples


		if (mymillis > 500) //if more than 500ms elapsed from last buffer record
		{
			buffer_to_i2s(); //execute last buffered sample

		}
          }
 
I can't figure out why it works when there's no delay because there are two problems that I see.
memcpy copies a specified number of bytes. You have specified the number of samples. Both instances of memcpy should be copying MAX_SAMPLES*2
The other problem is that the buffer array is too big. The T4 ram is split into two separate chunks of 256kB each and buffer is 128*1536*2 = 393,216 bytes
Since you're delay is 0.5 seconds, you only need the buffer to be at least 44100*0.5*2 = 44100 bytes which is 173 audio buffers.
Fix the memcpy problem and try a MAX_QUEUE_SIZE of, say, 200.

+edit - make that three (potential) problems. You don't need AudioMemory to be set to such a high value (256), which will use up memory that you need for the buffer. It should work perfectly well with 8, or even less.

Pete
 
...
The other problem is that the buffer array is too big. The T4 ram is split into two separate chunks of 256kB each and buffer is 128*1536*2 = 393,216 bytes
...

T4 RAM is two blocks of 512 KB. Low RAM1 block of 512KB holds code and compile allocs and runs at CPU Speed. Upper RAM2 holds DMA memory and dynamic allocs - and runs at CPU/4 speed.

See Memory Layout on : pjrc.com/store/teensy40.html
 
Ooops. My bad.
But the output of imxrt-size.exe:
Code:
ITCM :  18944 B	( 7.23% of  256 KB)
DTCM : 406208 B	(154.96% of  256 KB)
OCRAM:  79456 B	(15.16% of  512 KB)
Flash:  28720 B	( 1.41% of 1984 KB)
Ptext     =  18944
Init_data =   3376
UnIn_data = 402832
shows that the array has overflowed DTCM. Looks like there's room for it in OCRAM. I forget which directive puts it in there.

Pete
 
Ooops. My bad.
But the output of imxrt-size.exe:
Code:
ITCM :  18944 B	( 7.23% of  256 KB)
DTCM : 406208 B	(154.96% of  256 KB)
OCRAM:  79456 B	(15.16% of  512 KB)
Flash:  28720 B	( 1.41% of 1984 KB)
Ptext     =  18944
Init_data =   3376
UnIn_data = 402832
shows that the array has overflowed DTCM. Looks like there's room for it in OCRAM. I forget which directive puts it in there.

Pete

Pete:
Kurt posted an updated imxrt-size on his github. It gives better feedback - here is the last compile done here:
Code:
FlexRAM section ITCM+DTCM = 512 KB
    Config : aaaaaaab
    ITCM :  28752 B	(87.74% of   32 KB)
    DTCM :  12992 B	( 2.64% of  480 KB)
    Available for Stack: 478528
OCRAM: 512KB
    DMAMEM:  12384 B	( 2.36% of  512 KB)
    Available for Heap: 511904 B	(97.64% of  512 KB)
Flash:  38192 B	( 1.88% of 1984 KB)

ITCM and DTCM are in the same 512KB RAM1 block for Instructions and Data in OCRAM (on chip RAM).
 
I can't figure out why it works when there's no delay because there are two problems that I see.
memcpy copies a specified number of bytes. You have specified the number of samples. Both instances of memcpy should be copying MAX_SAMPLES*2
The other problem is that the buffer array is too big. The T4 ram is split into two separate chunks of 256kB each and buffer is 128*1536*2 = 393,216 bytes
Since you're delay is 0.5 seconds, you only need the buffer to be at least 44100*0.5*2 = 44100 bytes which is 173 audio buffers.
Fix the memcpy problem and try a MAX_QUEUE_SIZE of, say, 200.

+edit - make that three (potential) problems. You don't need AudioMemory to be set to such a high value (256), which will use up memory that you need for the buffer. It should work perfectly well with 8, or even less.

Pete

Thank you el_supremo, with your three suggestion the code works correctly.

I was convinced that samples and bytes had the same meaning.

Now i'd like to understand what's the difference between samples , audio packets and bytes.

Here's the code updated i'm currently using:

Code:
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>


AudioRecordQueue         queueRecord;
AudioPlayQueue           queuePlay;
AudioInputI2S            i2s1Record;
AudioAnalyzePeak         peak1;
//AudioAnalyzePeak         peak2;
AudioOutputI2S           i2s2Play;
AudioConnection          patchCord1(i2s1Record, 0, queueRecord, 0);
AudioConnection          patchCord2(queuePlay,  0, i2s2Play, 1);
AudioConnection          patchCord3(i2s1Record, 0, peak1, 0);
//AudioConnection          patchCord4(i2s1, 0, peak2, 1);
AudioControlSGTL5000     sgtl5000_1;


#define 		MAX_SAMPLES 	128
#define 		MAX_QUEUE_SIZE  200

uint16_t 		buffer[MAX_SAMPLES * MAX_QUEUE_SIZE];
int32_t			record_offset = 0;
int32_t  		play_offset = 0;


elapsedMillis 	mymillis;


// INPUT     OUTPUT
// i2s1 L >> i2s2 L
// i2s1 R >> i2s2 R

void setup()
{
  Serial.begin(115200);
  AudioMemory(8);
  sgtl5000_1.enable();
  sgtl5000_1.volume(0.5);
  sgtl5000_1.inputSelect(AUDIO_INPUT_MIC);
  sgtl5000_1.micGain(36);
  //sgtl5000_1.lineInLevel(0);
  sgtl5000_1.lineOutLevel(13);
  queueRecord.begin();
  mymillis = 0;
}

void printPeak()
{
	Serial.println(peak1.read());
	delay(5);
}


void i2s_to_buffer()
{
	memcpy(buffer + record_offset, queueRecord.readBuffer(), MAX_SAMPLES * 2);
	queueRecord.freeBuffer();
	record_offset+=MAX_SAMPLES;
	if (record_offset >= (MAX_SAMPLES * MAX_QUEUE_SIZE))	record_offset=0;
}

void buffer_to_i2s()
{
   memcpy(queuePlay.getBuffer(), buffer + play_offset , MAX_SAMPLES * 2);
    queuePlay.playBuffer();
    play_offset+=MAX_SAMPLES;
    if (play_offset >= (MAX_SAMPLES * MAX_QUEUE_SIZE)) play_offset=0;


}


void loop()
{

	//printPeak();
	//delay(1000);

	if(queueRecord.available() >= 2)
	{

		i2s_to_buffer(); // buffer samples


		if (mymillis > 500) //if more than 100ms elapsed from last buffer play
		{
			buffer_to_i2s(); //execute last buffered sample

		}

		//Serial.print(record_offset);
		//Serial.print(" ");
		//Serial.print(play_offset);
		//Serial.println();

	}// end if(queueRecord.available() >= 2)



}
 
- a byte is 8 bits.
- a sample is 16 bits = 2 bytes
- a audio packet is 128 samples = 256 bytes.

ok, so the i2s input fill the queue connected with audio packets
from the Notes of the queue input object:

Up to 52 packets may be queued by this object, which allows approximately 150 ms of audio to be held in the queue

so we have a max queue of 52 elements and every element contains an array of 128 elements of int16_t type, is it right?

The memcpy function copies the num of bytes specified in the last parameters so a x2 is required.

There are only two other things that are not clear to me:

  • Why a value of 8 was suggested?

From "Audio Connections & Memory" it says:

AudioMemory(numberBlocks);
Allocate the memory for all audio connections. The numberBlocks input specifies how much memory to reserve for audio data. Each block holds 128 audio samples, or approx 2.9 ms of sound. Usually an initial guess is made for numberBlocks and the actual usage is checked with AudioMemoryUsageMax().


So, why we are reserving just 23ms of data? Does this vector also refer to i2s input or output data?



  • In the playBuffer function of queue output, why the max queue size is 32?


Code:
void AudioPlayQueue::playBuffer(void)
{
	uint32_t h;

	if (!userblock) return;
	h = head + 1;
	if (h >= 32) h = 0;
	while (tail == h) ; // wait until space in the queue
	queue[h] = userblock;
	head = h;
	userblock = NULL;
}

Anyway thank you all for taking the time and clearing up most of my doubts.
This board is truly incredible and I think it is really useful to bring people to "touch" the audio world.
 
Status
Not open for further replies.
Back
Top