Line in audio disappears as soon as I initR a TFT display

K

kenjib

Member
Hello,

I have a Teensy 4.1 with the corresponding audio shield. I also have a TFT display but I don't even have to plug it in to demonstrate the behavior -- this happens with or without successfully connecting the display. Steps to reproduce:

1. Mount audio shield to Teensy 4.1.
2. Connect headphones to audio shield.
3. Connect a mono line to the line in (left) connections on the Audio Shield (in my case a 1/4" jack with a guitar plugged in).
4. Run the code below. I can hear the audio signal passing through the Teensy fine in the headphones.
5. Uncomment the initR line (line 34).
6. Now I can no longer hear the audio signal from line in left. The audio in signal suddenly disappears.

Line out still works -- I tested that using different code and it is fine when outputting audio from a synth build in conjunction with the display running. I can output audio to the headphone jack fine but there is no audio coming in. I also tried changing the pin numbers for DC and CS and it was the same. Again, it happens whether or not the display is even connected.

Any thoughts/help/suggestions would be greatly appreciated!

Code: 
#include <ST7735_t3.h>


#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// GUItool: begin automatically generated code
AudioInputI2S            i2s1;           //xy=215,105
AudioOutputI2S           i2s2;           //xy=975,107
AudioConnection          patchCord1(i2s1, 0, i2s2, 0);
AudioConnection          patchCord2(i2s1, 1, i2s2, 1);
AudioControlSGTL5000     sgtl5000_1;     //xy=609,180
// GUItool: end automatically generated code


const int PIN_DISPLAY_CLK = 13;
const int PIN_DISPLAY_MOSI = 11;
const int PIN_DISPLAY_DC = 9;
const int PIN_DISPLAY_RST = 8;
const int PIN_DISPLAY_CS = 10;


void setup() {
  AudioMemory(256);

  sgtl5000_1.enable();
  sgtl5000_1.volume(0.80);
  sgtl5000_1.inputSelect(AUDIO_INPUT_LINEIN);

  ST7735_t3 tft = ST7735_t3(PIN_DISPLAY_CS, PIN_DISPLAY_DC, PIN_DISPLAY_MOSI, PIN_DISPLAY_CLK, PIN_DISPLAY_RST);
//  tft.initR(INITR_BLACKTAB);
}

void loop() {
}
 
kenjib said:
Any thoughts/help/suggestions would be greatly appreciated!
Click to expand...

According to <<this>> webpage, the Rev D Audio Adapter Board uses pin 8 as "DOUT (IN) Audio Data from Audio Shield to Teensy". When you initialize your display, the initialization is very likely changing pin 8 (RESET) from INPUT mode needed by the Audio Adapter to OUTPUT mode used to RESET the display.

Hope that helps . . .

Mark J Culross
KD5RXT
 
Thanks Mark. That's spot on.

I was able to get it all working! Now here seems like a weird little thing I found. I added a record queue and a playback queue. If I call AudioRecordQueue.begin before initR there is a 1/2 second delay in the audio output. If I just switch the order so that begin is after, the delay is gone. Is this just the documented issue for AudioRecordQueue?

After calling begin, readBuffer() and freeBuffer(), or clear() must be used frequently to prevent the queue from filling up.
Click to expand...

Again, I can fix this by making sure the call to begin() is the last thing that happens in setup. So now I am thinking about how this can effect latency overall. I am thinking that what is going on is that the audio data gets backed up in the AudioPlayQueue buffer. That means that whatever time gap there is between begin() getting called and the first buffer being sent to the play buffer gets permanently added to the application latency. Would it make sense, then, to handle this by -- for example -- putting something in the loop code that, the first time through only, reads all of the existing AudioRecordQueue queue items and tosses them out so that the whole process of going to the play buffer has as little latency as possible?

Is there any delay between the end of setup and the first time loop is called? If not then this would be pretty much moot I suppose and the issue can be avoided by just making sure that the call to begin is just always the very last thing in the setup function.

Code below:

Code: 
#include <ST7735_t3.h>


#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// GUItool: begin automatically generated code
AudioInputI2S            i2s1;           //xy=215,105
AudioRecordQueue         queue1;         //xy=397,108
AudioPlayQueue           queue2;         //xy=630,114
AudioOutputI2S           i2s2;           //xy=975,107
AudioConnection          patchCord1(i2s1, 0, queue1, 0);
AudioConnection          patchCord2(queue2, 0, i2s2, 0);
AudioControlSGTL5000     sgtl5000_1;     //xy=609,180
// GUItool: end automatically generated code


const int PIN_DISPLAY_CLK = 13;
const int PIN_DISPLAY_MOSI = 11;
const int PIN_DISPLAY_DC = 36;
const int PIN_DISPLAY_RST = 37;
const int PIN_DISPLAY_CS = 38;


void setup() {
  AudioMemory(512);

  sgtl5000_1.enable();
  sgtl5000_1.volume(0.80);
  sgtl5000_1.inputSelect(AUDIO_INPUT_LINEIN);

  queue1.begin();

  ST7735_t3 tft = ST7735_t3(PIN_DISPLAY_CS, PIN_DISPLAY_DC, PIN_DISPLAY_MOSI, PIN_DISPLAY_CLK, PIN_DISPLAY_RST);
  tft.initR(INITR_BLACKTAB);

  // If I move queue1.begin() here, the latency disappears.
}

void loop() {
  if (queue1.available() >= 1) {
    int16_t *audioBuffer = queue2.getBuffer();
    memcpy(audioBuffer, queue1.readBuffer(), 256);
    queue1.freeBuffer();
    queue2.playBuffer();
  }
}
 
A quick test shows 0 milliseconds from end of setup to start of loop so I'll just make sure that begin() is always last. Good to know!
 
I have a little more info now on the interactions between this display and the record/playback audio queues. My processing loop is as follows:

Code: 
void loop() {
  Effect *effect1 = window.getEffect(0);
  // processAudioEngine needs to be first. See above notes.
  processAudioEngine(effect1);
  if (effect1->usesSynthMidi) {
    processMidi(usbMIDI);
  }

  static int i = 0;
  i++;
  if (i == 1000) {
    i = 0;
    pollInput();
  }  
  
  window.render();
}

render() only does stuff as necessary, so in my test case it is only running the first time. processAudioEngine is where the queues are handled. I'm only using code excerpts for now to illustrate because the project is pretty big.

Code: 
AudioRecordQueue         effect1In;      //xy=265,66
AudioPlayQueue           effect1Out;     //xy=402,65

void processAudioEngine(Effect * effect1) {
  if (effect1In.available() >= 1) {
    int16_t *audioBuffer = effect1Out.getBuffer();
    memcpy(audioBuffer, effect1In.readBuffer(), 256);
    effect1In.freeBuffer();
    effect1->processEffect(audioBuffer);
    effect1Out.playBuffer();
  }
}

The effects and "processEffect" are there to apply a selectable audio effect to the sound and aren't related this. The first time through the loop, the queues are fine. However, at the end I render the windows, which introduces another delay to process. So during the second time through the loop, the playback buffer again gets backed up and never recovers. In this case I measured it as 16 buffers backed up, which makes the latency somewhere around 48ms, which starts to become very noticeable. Furthermore, each time I need to re-render the UI I will possibly get backed up up to another 16 buffers, depending on how much needs to be rendered. Polling for input and responding to MIDI don't seem to be causing issues yet.

My temporary solution is to always throw away extra buffers to catch up:

Code: 
AudioRecordQueue         effect1In;      //xy=265,66
AudioPlayQueue           effect1Out;     //xy=402,65

void processAudioEngine(Effect * effect1) {
  if (effect1In.available() >= 1) {
[B][I]    while (effect1In.available() > 1) {
      Serial.print(effect1In.available());
      effect1In.readBuffer();
      effect1In.freeBuffer();
    }
[/I][/B]    int16_t *audioBuffer = effect1Out.getBuffer();
    memcpy(audioBuffer, effect1In.readBuffer(), 256);
    effect1In.freeBuffer();
    effect1->processEffect(audioBuffer);
    effect1Out.playBuffer();
  }
}

This works to fix latency, but will make the audio glitch/pop every time I update the UI, which long term isn't a good solution. Since I haven't found a way to multithread or interleave the display updates, it's looking like I will probably eventually need to move the UI and input to a second processor. Is there a way to have this run smoothly on one processor without throwing away audio buffers every time the UI updates?
 
You must log in or register to reply here.
Back
Top