Bugs with Queues

SteveMcI

Member
Hi everyone. I wanted to share a few things I’ve noticed while working on a project utilizing the Teensy 4.1 and the Audio Board Rev D. Before I begin, I do want to say I generally love the Audio.h library and have used it for a number of projects, but I have found a few limitations I wanted to share with the community.

Overall Project Summary:
I am building an electrophysiological (ephys) device to measure the Auditory Brainstem Response. This is a test that measures hearing and primarily used to test the hearing of subjects who cannot behaviorally respond, aka babies and animals. The necessary components of this is to stimulate with a short tone pip, at a specific loudness (dB), and record the evoked response from electrodes on the head. The response is averaged from multiple repetitions of the stimulus (usually between 512-1024).

I am approaching this project using a Teensy 4.1, audio board rev D, a modified EEG click ephys preamp (EEG click is not needed to test the issues below). Teensy stimulates and records, then sends information via the serial monitor to MATLAB where I will do most of my analysis.
I plan on doing a full write up once I get everything working and tested.

Stimulus (working)
Currently I have the stimulus. Here I use premade .wav files on the SD card to play. The wav files themselves are stereo; the left channel is the stimulus itself, a tone of some frequency and duration, while the right channel is a trigger so that I can more easily align all the averages. Additionally, I have figured out the calibration for loudness and can accurate play out tone of a specific dB (not in the example code below).

Acquisition (issues)
I am now working on the acquisition side and noticed a few things I wanted to share.

Issue 1) I posted on this before but to reiterate, the signal from the waveform is picked up by the line in even if it is physically disconnected. Although it is significantly attenuated, it is still there (+/-850 bits disconnected vs +/- 32,768 bits plugged in). This seems to be fixed if I remove the “audioboard.volume(0.8)” from my setup() function (figure 1, pay attention to y-axes). However, removing this line prevents me from using the headphone output effectively.

Issue 2) Since I need to stream incoming analog signal, I am reading out from the queues in the audio.h library. The queue is made up of ‘packets’ of approximately 2.9 ms of audio (at 44.1kHz). I have noticed that the queues ‘pop’ at the beginning (or end) or each packet. This is made worse when trying the inline in to ground (figure 2). Atleast I believe this pop is originating from the queue because the pops appear every 2.9 ms (figure 3).



I’m still working on solutions to these issues. I am currently playing with additional ADC boards using the I2S2 port on the Teensy 4.1. Right now I am playing with the PCM1802 board and following the direction posted by Paul here https://www.pjrc.com/pcm1802-breakout-board-needs-hack/. I like this board because of the 24bit resolution but if I use it, I want to use all 24bits and the Audio.h library however can only handle 16bits. I have no experience of modifying libraries so I am looking into this. Even if I do find a way to do this, I’m afraid the popping (issue 2) will still be present. If any one has any suggestions I would love some feedback.



Below is simplified Teensy and MATLAB code I used to test these issues. Also attached are the example .wav files. Hardware wise, I only used a Teensy 4.1 and Audio Board Rev D. The lineInR and lineOutR are tied together with a jumper cable (this is for the trigger). LineInL is tied to GND, lineOutL, or nothing depending, see figure 1-2.


Teensy Code
Code:
#include <Bounce.h>
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// GUItool: begin automatically generated code
// INPUT connections
AudioInputI2S         lineIN;
AudioRecordQueue      queue_L;
AudioRecordQueue      queue_R;
AudioConnection       patchCord1(lineIN, 0, queue_L, 0);
AudioConnection       patchCord2(lineIN, 1, queue_R, 0);
// OUTPUT connections
AudioPlaySdWav        playWav;
AudioOutputI2S        audioOutput;
AudioAmplifier        ampL;
AudioAmplifier        ampR; // this amplifier will not be modified (always be x1) but is needed to maintain the temporal accuracy of the left and right channels
AudioConnection       patchCord3(playWav, 0, ampL, 0);
AudioConnection       patchCord4(ampL, 0, audioOutput, 0);
AudioConnection       patchCord5(playWav, 1, ampR, 0);
AudioConnection       patchCord6(ampR, 0, audioOutput, 1);
AudioControlSGTL5000  audioboard;

// define pins for audio shield
#define SDCARD_CS_PIN 10
#define SDCARD_MOSI_PIN 7
#define SDCARD_SCK_PIN 14

// Variables for audio queue
int fs = 44100;                           // not used - sample rate used by audioboard
const int numPackets = 18;                // How many packets to read
const int bufferleng = 128 * numPackets;  // each packet is 128 samples long, approximately 2.9 ms duration

void setup() {

  //Serial communication
  Serial.begin(230400);
  delay(300);  //wait for init of serial
  while (!Serial.available()) {
    //do nothing and wait for handshack
  }
  Serial.println("CONNECTED");

  // Audio connections require memory to work. define here
  AudioMemory(80);

  // Enable the audio shield and set the output volume.
  audioboard.enable();
  audioboard.inputSelect(AUDIO_INPUT_LINEIN);
  audioboard.volume(0.8);

  // Setup SD card
  SPI.setMOSI(SDCARD_MOSI_PIN);
  SPI.setSCK(SDCARD_SCK_PIN);
  if (!(SD.begin(SDCARD_CS_PIN))) {
    //make sure accessing SD
    while (1) {
      Serial.println("Unable to access the SD card");
      delay(500);
    }
  }
  
  ampL.gain(1);
  ampR.gain(1);

  Serial.print("Num packets = ");
  Serial.println(numPackets);

  Serial.println("START");
}


// Main loop
void loop() {

  if(Serial.available()){
          
    // setup local variables
    int16_t data_L[bufferleng];
    int16_t data_R[bufferleng];
    bool done = 0;
    char incoming = Serial.read();
    
    //play tone
    if ((incoming == 1) || (incoming == 2)){
        
      if(incoming == 1) {
        playWav.play("1000Hz_tone.wav");
      } else if (incoming == 2){
        playWav.play("silence_tone.wav");
      }
    
      delay(1); // usually takes a ms for the stimulus to start playing

      // begin queue for recording data
      queue_L.begin();
      queue_R.begin();

      if (playWav.isPlaying()){

        //  wait till queue has enough packets then stop queue and stop playing
        while(!done){

          if ((queue_L.available() >= numPackets) && (queue_R.available() >= numPackets)) {
            queue_L.end();
            queue_R.end();

            playWav.stop();

            done = 1;
          }
        }
      }

      // *********** wait till after playing to organize data and send to Serial.print *************

      for (int i = 1; i <= numPackets; i++){
        uint16_t (*buffer_L) = (queue_L.readBuffer());
        queue_L.freeBuffer();
        uint16_t (*buffer_R) = (queue_R.readBuffer());
        queue_R.freeBuffer();

        for (int j = 0; j < 128; j++){
          data_L[128*(i-1) + j] = buffer_L[j];
          data_R[128*(i-1) + j] = buffer_R[j];
        }
      }

      // clear queue
      queue_L.clear();
      queue_R.clear();

      // Serial print data with ear info
      Serial.print("L,");
      for (int i = 0; i < bufferleng; i++){      
        Serial.print(data_L[i]);
        Serial.print(",");
      }
      Serial.println();

      // Serial print trigger
      Serial.print("R,");
      for (int i = 0; i < bufferleng; i++){      
        Serial.print(data_R[i]);
        Serial.print(",");
      }
      Serial.println();
      
    }  
  }
  // for stability
  delay(10);
}


MATLAB (2022b) code
Code:
%% Setup

clear all
% close all
clc

% % list serial ports
% serialportlist("all")

% connect to Teensy - Make sure Serial Port is Correct
% timeout after 1 second
a=serialport("COM6", 230400, 'Timeout', 1);

write(a, 10, "char"); pause(0.01); % write something (enter key) to Serial Monitor to start system

numpackets = 0;

while 1
    % read in from teensy
    dat=convertStringsToChars(readline(a));

    % see if data is done
    if ~isempty(dat)
        if contains(dat, 'START')
            break
        elseif contains(dat, 'Num packets = ')
            numpackets = str2num(dat(15:end-1));
        elseif contains(dat, "Unable to access the SD card")
            error('UNABLE TO READ SD CARD')
        end
    else
        disp('empty')
    end
end

if numpackets == 0
    error('number of packets not declared')
end

disp('READY')

%% Run

numAvg = 20; % how many times to repeat the same stimulus

sr = 44100;
nsamp = numpackets * 128; % number of queue packets times number of samples per packet
time = (0:1/sr:(nsamp-1)*1/sr)*1000-5;
[~, zeroIDX] = min(abs(time-0));
time = time - time(zeroIDX);

in_min = -32768;
in_max = 32768;
out_min = -1.65;
out_max = 1.65;

dataL = nan(numAvg, nsamp);
dataR = nan(numAvg, nsamp);

figure;

for i = 1:numAvg

    clearvars -except a dataL dataR time in_min in_max out_min out_max numAvg i sr numpackets zeroIDX

    if 0 % tell teensy to play file 1000Hz_tone.wav
        write(a, 1, "char");  pause(0.01);
        disp(['rep = ', num2str(i), ' : playing 1000Hz_tone.wav'])
    else % tell teensy to play file silence_tone.wav
        write(a, 2, "char");  pause(0.01);
        disp(['rep = ', num2str(i), ' : playing silence_tone.wav'])
    end

    % gather data
    dataL_raw = convertStringsToChars(readline(a));
    dataR_raw = convertStringsToChars(readline(a));

    % catch to make sure incoming data makes sense
    if (contains(dataL_raw(1), 'L') && contains(dataR_raw(1), 'R'))

        dataL_samp = str2num(dataL_raw(2:end));
        dataR_samp = str2num(dataR_raw(2:end));

        % convert to volts and round trigger
%         dataL_volts = (dataL_samp - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
        dataR_volts = round((dataR_samp - in_min) * (out_max - out_min) / (in_max - in_min) + out_min);
        dataL_volts = dataL_samp;
%         dataR_volts = dataR_samp;

        dataL_aligned = zeros(size(time));
        dataR_aligned = zeros(size(time));

        % align triggers and wav
        mx = max(dataR_volts);
        startIDX = find(dataR_volts>mx/2, 1);
        if startIDX > zeroIDX
            x = dataL_volts((startIDX - zeroIDX):end);
            dataL_aligned(1:length(x)) = x;
            x = dataR_volts((startIDX - zeroIDX):end);
            dataR_aligned(1:length(x)) = x;
        else
            x = [nan(1,abs(startIDX - zeroIDX)), dataL_volts(1:end-abs(startIDX - zeroIDX))];
            dataL_aligned(1:length(x)) = x;
            x = [nan(1,abs(startIDX - zeroIDX)), dataR_volts(1:end-abs(startIDX - zeroIDX))];
            dataR_aligned(1:length(x)) = x;
        end

        % convert to volts
        dataL(i,:) = dataL_aligned;
        dataR(i,:) = dataR_aligned;

        subplot(2,1,1);
        title('wave')
        plot(time, mean(dataL,'omitnan'))
        subplot(2,1,2);
        title('trigger')
        plot(time, mean(dataR,'omitnan'))

    end

    pause(0.05)
end
 

Attachments

  • Figure 3.pdf
    773.3 KB · Views: 28
  • Figure 2.pdf
    795.4 KB · Views: 24
  • Figure 1.pdf
    800.1 KB · Views: 38
  • Wav Files.zip
    1.8 KB · Views: 24
I suspect many of the issues are hardware. To prove this to yourself, feed the queues with a synthesised signal using AudioWaveform - you should see no pops apart from any discontinuities cause by starting and stopping the waveforms. I still don’t like your practice of freeing the queued data back to the audio system before you use it, though…

A 1ms delay is not enough to be sure the playback has started, you need at least 3ms to be sure an audio update has occurred. Better yet is to start the queues at the same time as playback, and wait until they both have a packet available, you’re then 100% sure an audio update has happened. Up to you whether you discard those packets or not.

The clicks and low-level residual audio appear to me to be likely due to crosstalk between the headphone output, CPU activity, and the line inputs. You could perhaps try a separate headphone amplifier wired to the line out, and supplied from the 5V, so you’re not driving that low impedance load from the audio board’s supply. There’s not a lot can be done about CPU activity spikes, apart from making sure your wiring is well done with good solder joints.

For 24-bit audio within the confines of the Audio library and your use case, you could look into modifying the 2-channel I2S input by adding a couple of extra outputs, and put the bottom 8 bits of each channel out on outputs 2 and 3. I think examination of the code will show it’s discarding data, though not sure about that.
 
Thanks for the response.

A 1ms delay is not enough to be sure the playback has started, you need at least 3ms to be sure an audio update has occurred. Better yet is to start the queues at the same time as playback, and wait until they both have a packet available, you’re then 100% sure an audio update has happened. Up to you whether you discard those packets or not.
agreed but this doesn't really change anything

The clicks and low-level residual audio appear to me to be likely due to crosstalk between the headphone output, CPU activity, and the line inputs. You could perhaps try a separate headphone amplifier wired to the line out, and supplied from the 5V, so you’re not driving that low impedance load from the audio board’s supply. There’s not a lot can be done about CPU activity spikes, apart from making sure your wiring is well done with good solder joints.
Unfortunate but I've known about this issue for some weeks now and found I can work around it. It would be nice if it wasn't the case but I can survive.

I suspect many of the issues are hardware. To prove this to yourself, feed the queues with a synthesised signal using AudioWaveform - you should see no pops apart from any discontinuities cause by starting and stopping the waveforms.
You are correct, when I synthesize a waveform there are no longer pops. Unfortunately I found that this approach makes it hard to have full control over the stimulus waveform, plus it's hard to visualize. I need to be able to control the frequency, duration, envelop (on/off ramp or other), phase, and repetition rate. I've attached a figure of some example waveform stimuli I use. While not impossible, I found this approach very cumbersome. It was much easier to create my stimuli in MATLAB, visualize, and save as .wav file.

Thanks

View attachment Figure 4.pdf
 
agreed but this doesn't really change anything
I believe you’ll find it does. Because you check for the playing state too soon, about 2/3 of the time it’ll completely skip the while() loop, fail to stop the queues, and emit garbage data or just crash because you don’t check for a null pointer being returned from readBuffer().

I could be wrong, this is just based on code inspection, not building and running it.
You are correct, when I synthesize a waveform there are no longer pops. Unfortunately I found that this approach makes it hard to have full control over the stimulus waveform, plus it's hard to visualize. I need to be able to control the frequency, duration, envelop (on/off ramp or other), phase, and repetition rate. I've attached a figure of some example waveform stimuli I use. While not impossible, I found this approach very cumbersome. It was much easier to create my stimuli in MATLAB, visualize, and save as .wav file.
I wasn’t intending to suggest you change your approach, just narrowing down the causes of the issues you described. Since you now know that real time playback directly from the SD card is one such cause, and it appears the stimulus files are fairly short, it becomes immediately apparent how to avoid the issue. I’ll leave that as an Exercise for the Reader…
 
Oh, I just looked at the I2S code. It’s set for 16-bit acquisition, so the edits needed to get pseudo 24-bit are a bit more extensive than I’d hoped :(
 
Back
Top