ESP32 as I2S input-device

[SomeoneFromGermany]

Hello,

I want to use an ESP32 as an bluetooth audio reciver.
I found this https://github.com/dvxlab/esp32_bt_music_receiver library which is easy to use and can strem audio directly to the ESP32's I2S connection.

In theorie it should work, but it didn't.

The reason is that the ESP32 sends data not synced to the clock.
The BCLK from the teensy is at 2.8 MHz, but it should be (44.1kHz * 16 * 2) 1.41 MHz.



How can I fix this issue?

Teensys code:

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// GUItool: begin automatically generated code
AudioInputI2S            i2s1;           //xy=246,239.00000190734863
AudioOutputAnalogStereo  dacs1;          //xy=369.00000381469727,239.00000381469727
AudioConnection          patchCord1(i2s1, 0, dacs1, 0);
AudioConnection          patchCord2(i2s1, 1, dacs1, 1);
// GUItool: end automatically generated code

void setup() {
  AudioMemory(20);
}

void loop() {
  
}

ESP32's code

#include <arduino.h>
#include "esp32_bt_music_receiver.h"

BlootoothA2DSink a2d_sink;

void setup() {
  a2d_sink.start("MyMusic");
  Serial.begin(115200);
}

unsigned long last = 0;

void loop() {
  if ((millis() - last) > 100) {
    last = millis();
    Serial.println(a2d_sink.get_audio_state());
  }
}

The I2S config

i2s_config = {
    .mode = (i2s_mode_t) (I2S_MODE_SLAVE | I2S_MODE_TX),
    .sample_rate = 44100,
    .bits_per_sample = (i2s_bits_per_sample_t)16,
    .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
    .communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
    .intr_alloc_flags = 0, // default interrupt priority
    .dma_buf_count = 8,
    .dma_buf_len = 64,
    .use_apll = false
  };

[PaulStoffregen]

The BCLK from the teensy is at 2.8 MHz, but it should be (44.1kHz * 16 * 2) 1.41 MHz.

The I2S protocol allows any number of bits per clock phase. Long ago (early days of Teensy 3.0 & 3.1) we used the minimum 16 bits per clock phase, as this code expects. Many chips worked fine, but it turned out many others would only work with 32 bits per clock phase (where the lowest 8 or 16 bits are ignored padding bits). Virtually all MEMS microphones with I2S output require 32 bits per phase. We switched to the 32 bits per clock phase format, because it worked with all I2S chips.

Well, it worked with everything until now.

You have a few options....

Maybe try using I2S master mode on the ESP side and I2S slave mode on Teensy. But to be honest, all testing in recent years has been done with 32 bits per clock phase, so I can't say whether Teensy's I2S slave mode code properly receives 16 bits per phase. It should, so if you find it doesn't, please report it and I'll add to a list of known issues. Just to be realistic, odds it would ever get fixed are low, since this format is so uncommon and so many people want so many other features! But there's a chance it might just work, so try this first.

Maybe you could configure the ESP side to transmit 32 bits per clock phase, in either master or slave moce? I don't know much about ESP and the code used, but I can tell you this is by far the most commonly used format. Again, many chips only work with that specific format, especially MEMS microphones. If the ESP hardware can do it, seems likely the software support would be there.

As a last resort, you could look for the ancient audio library code on github which used 16 bits per phase. That was long before Teensy 4.0 and 4.1, but the SAI hardware is pretty much identical in the newer boards, so if you are using those most of the porting work would involve the very different clocks. But this will require diving in the code and doing a lot of low-level experimentation, so I wouldn't recommend this route unless all else fails.

[SomeoneFromGermany]

Maybe try using I2S master mode on the ESP side and I2S slave mode on Teensy.

This could work for my example, but for my project i need the teensy as a master, because i want to use the audioshield simultaneously with the ESP32 in quad-channel.

Maybe you could configure the ESP side to transmit 32 bits per clock phase, in either master or slave moce?

Works.... But with a lot of noise and distorted audio.
I'm trying this on a breadboard. Maybe soldering works better. But i dont think it will work so much better.
(And not using the DAC's unfiltered)

[SomeoneFromGermany]

I tried quad channel with an audio shield, but the audio is still very bad.

I don't know if this is an soft- or hardware problem but without I2S data the audioshield outputs 27 to 100kHz signal and if audio is played its distorted.

Orange: ESP -> Teensy
Green: Teensy -> Audioshield

[PaulStoffregen]

Is that blue waveform BCLK?

I'm not sure how to read the info from this screen. It doesn't quite look like any oscilloscope I know. The scale is shown at 465mV. If that really means 0.465 volts per division (very unusual) and the waveform looks like maybe 2.5 divisions tall, that would be about 1.2 volts peak-to-peak voltage.

Teensy uses 3.3V signals. I believe most ESP chips use 3.3V too, though I'm not an expert with ESP.

Is it possible both chips are trying to drive the same BCLK signal?

[SomeoneFromGermany]

Its 500mV per division. (Oszilloscope is an Rohde & Schwarz RTB2004)

The BCLK is only transfered to the ESP32. I checked this at first.
The yellow waveform in the first image is the BCLK with around 3.3V p-p.

The blue wave is the output of the audioshield lineout. Its around +/- 750mV.

The digital audio signal from the ESP to the teensy looks right, but I could not figure out how i can look at the .SET waveform file to look at the waveform in more detail.

Could it be a problem that I set the ESP from 16 to 32 bit and it messes something up, when it receives only 16 bit, with the MSB and/or LSB?
The ESP32 is an 32bit microcontroller, so it should have no problems working with 32 bit audio.

[JayShoe]

I've also tried this. I'm very keen on following and learning how to make this work. Maybe this thread will help? One user said he got it working, some code is in the the thread below.

https://forum.pjrc.com/threads/57301...-Stream-to-I2S

[SomeoneFromGermany]

It also works for me at this point. But only with the DAC's of my test teensy 3.6. There is this high-pitch noise. Not the same when using a passthroug USB->DAC's. My best guess at this point is that the ESP messes something up with the MSB and/or LSB, as in the code of alex6679 has a definition for nobits with the size 16. Which gets OR'red with a few Values.

uint8_t noBitsM1=noBits-1;  //noBits=16

How can the teensy be set to ignore the first 16bit? Maybe this would get rid of this.

[JayShoe]

How can the teensy be set to ignore the first 16bit? Maybe this would get rid of this.

If I'm not mistaken, Teensy does ignore the first 16bits (by default), so maybe that code you quoted is the key?

Long ago (early days of Teensy 3.0 & 3.1) we used the minimum 16 bits per clock phase, as this code expects. Many chips worked fine, but it turned out many others would only work with 32 bits per clock phase (where the lowest 8 or 16 bits are ignored padding bits).

[SomeoneFromGermany]

I changed my test-setup to a teensy 4.1 and soldered a test-PCB for better testing, because in the audio design GUI its noted that
"On Teensy 3.x, the BCLK/LRCLK ratio is 32, which is not compatible with most MEMS microphones. Teensy 4.x uses BCLK/LRCLK ratio, which can be used with I2S MEMS microphones."

I also change both the code of the ESP and teensy.
ESP:

#include <arduino.h>
#include "esp32_bt_music_receiver.h"

BlootoothA2DSink a2d_sink;

void setup() {
  static const i2s_config_t i2s_config = {
    .mode = (i2s_mode_t) (I2S_MODE_SLAVE | I2S_MODE_TX),
    .sample_rate = 44100,
    .bits_per_sample = (i2s_bits_per_sample_t)32,
    .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
    .communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
    .intr_alloc_flags = 0, // default interrupt priority
    .dma_buf_count = 8,
    .dma_buf_len = 64,
    .use_apll = false
  };


  a2d_sink.set_i2s_config(i2s_config);
  a2d_sink.start("MyMusic");
  Serial.begin(115200);
}

unsigned long last = 0;

void loop() {
  if ((millis() - last) > 100) {
    last = millis();
    Serial.println(a2d_sink.get_audio_state());
    //Serial.println(a2d_sink.get_audio_type());
  }
}

Teensy:

#include <Audio.h>
#include <Wire.h>

AudioInputI2SQuad        i2s_quad1;      //xy=172.00000381469727,224.00000762939453
AudioAnalyzeFFT1024      fft1024_1;      //xy=327.00000762939453,369.0000114440918
AudioAnalyzePeak         peak2;          //xy=331.0000114440918,333.00000953674316
AudioAnalyzePeak         peak1;          //xy=333.00000762939453,292.00000858306885
AudioOutputI2S           i2s1;           //xy=383,217
AudioConnection          patchCord1(i2s_quad1, 2, i2s1, 0);
AudioConnection          patchCord2(i2s_quad1, 2, peak1, 0);
AudioConnection          patchCord3(i2s_quad1, 2, fft1024_1, 0);
AudioConnection          patchCord4(i2s_quad1, 3, i2s1, 1);
AudioConnection          patchCord5(i2s_quad1, 3, peak2, 0);
AudioControlSGTL5000     sgtl5000_1;     //xy=302,184

void setup() {
  AudioMemory(20);
  sgtl5000_1.enable();
  sgtl5000_1.volume(0.7);
  Serial.begin(115200);
}

elapsedMillis a = 0;
void loop() {
  if(a >= 10 and peak1.available()){
    a = 0;
    Serial.println(peak1.read(), 20);
  }
}

To make things easier.

By reading the Peakof the incomming audio signal its clear that the audio is wrong interpreted by the teensy, because there also is this noise:

So changing

.communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),

to

.communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_LSB),

on the ESP's side, might point to something like that, because then the teensy receives no audio at all.

[SomeoneFromGermany]

I found the setup alex6679 was refering to:

	I2S1_RMR = 0;
	//I2S1_RCSR = (1<<25); //Reset
	I2S1_RCR1 = I2S_RCR1_RFW(1);
	I2S1_RCR2 = I2S_RCR2_SYNC(rsync) | I2S_RCR2_BCP  // sync=0; rx is async;
		    | (I2S_RCR2_BCD | I2S_RCR2_DIV((1)) | I2S_RCR2_MSEL(1));
	I2S1_RCR3 = I2S_RCR3_RCE;
	I2S1_RCR4 = I2S_RCR4_FRSZ((2-1)) | I2S_RCR4_SYWD((32-1)) | I2S_RCR4_MF
		    | I2S_RCR4_FSE | I2S_RCR4_FSP | I2S_RCR4_FSD;
	I2S1_RCR5 = I2S_RCR5_WNW((32-1)) | I2S_RCR5_W0W((32-1)) | I2S_RCR5_FBT((32-1));

in "C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Audio\ output_i2s.cpp" line 450. But configuring it like he said: "In config_i2s you would need to replace 32 with 16 at I2S_RCR4_SYWD((32-1)), I2S_RCR5_WNW((32-1)) and I2S_RCR5_W0W((32-1)) (not at I2S_RCR5_FBT((32-1))", just causes the audioshield to not work at all and the peak1 reads the same thing as before.

[alex6679]

Hi, maybe I can help you with your problem. I have a working setup with an ESP32 as bluetooth receiver and a T4 that receives the sound from the ESP32 via I2S. I don't have an audio shield and right now I also don't use the Teensy Audio library. If you want, I could try to compile a program that works with the audio library. I could test it with the spdif output and visualize the audio waveforms with the serial plotter of the Arduino IDE. This program should (theoretically) also work for you, and you could then replace the spdif output with the audio shield.

[alex6679]

I just noticed that you use AudioInputI2SQuad. Do you try to receive 4 audio channels from the ESP32?

[SomeoneFromGermany]

No, im trying to receive 2 channel audio from the ESP32 and an additional 2 channels from the audioshield.

Do you think my theorie with the MSB/LSB could be the key?

[alex6679]

I don't think that the problem is caused by the bit numbering. I use MSB and it works. Here is my I2S configuration on the ESP32

i2s_config_t i2s_config = {
        .mode = I2S_MODE_MASTER | I2S_MODE_TX,                                  // Only TX
        .sample_rate = 44100,
        .bits_per_sample = 16,
        .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,                           //2-channels
        .communication_format = I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB,
        .dma_buf_count = 6,
        .dma_buf_len = 128, //60
        .use_apll = 1,
        .intr_alloc_flags = 0,                                                  //Default interrupt priority
        .tx_desc_auto_clear = true                                              //Auto clear tx descriptor on underflow
    };

and at the T4

	uint8_t noBitsM1=noBits-1;     //noBits=16 in case of the ESP32
	I2S1_RCR4 = I2S_RCR4_FRSZ(1)
				| I2S_RCR4_SYWD(noBitsM1)
				| I2S_RCR4_MF
				| I2S_RCR4_FSE
				| I2S_RCR4_FSP;
	I2S1_RCR5 = I2S_RCR5_WNW(noBitsM1)
				| I2S_RCR5_W0W(noBitsM1)
				| I2S_RCR5_FBT(31);

Maybe I have some time this evening to try out the esp32 bluetooth sink code, that you posted above. Let's see if my setup works with your code. The program, that is runs on my ESP32, doesn't compile since the last update of the ESP_IDF. So I need to fix that anyway. Moreover, your example is much shorter.

[SomeoneFromGermany]

I tried your I2S configuration, but there was again no sound nor readable peak at the teensy.
The clock and the audio-data doesn't mach like in one of the first pictures i posted in this thread.

[BriComp]

The clock and the audio-data doesn't mach like in one of the first pictures i posted in this thread.

What do you mean? mach - going at the speed of sound!!

[SomeoneFromGermany]

match*
Like this:

[alex6679]

Ok, I got your example code on the ESP32 working. I googled for the esp32_bt_music_receiver.h header and found this bluetooth receiver class: https://github.com/dvxlab/esp32_bt_music_receiver. So I assume you also use this class? Anyway I use this class in combination with the following main:

#include <arduino.h>
#include "esp32_bt_music_receiver.h"

#define CONFIG_EXAMPLE_I2S_LRCK_PIN 23
#define CONFIG_EXAMPLE_I2S_BCK_PIN 22
#define CONFIG_EXAMPLE_I2S_DATA_PIN 19

BlootoothA2DSink a2d_sink;

void setup() {
 
  static const i2s_config_t i2s_config = {
    .mode = (i2s_mode_t) (I2S_MODE_MASTER | I2S_MODE_TX),
    .sample_rate = 44100,
    .bits_per_sample = (i2s_bits_per_sample_t)16,
    .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
    .communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
    .intr_alloc_flags = 0, // default interrupt priority
    .dma_buf_count = 8,
    .dma_buf_len = 128,//64,
    .use_apll = true,
    .tx_desc_auto_clear = true                                              //Auto clear tx descriptor on underflow
  };
    i2s_pin_config_t pin_config = {
      .bck_io_num = CONFIG_EXAMPLE_I2S_BCK_PIN,
      .ws_io_num = CONFIG_EXAMPLE_I2S_LRCK_PIN,
      .data_out_num = CONFIG_EXAMPLE_I2S_DATA_PIN,
      .data_in_num = -1                                                       //Not used
  };


  a2d_sink.set_i2s_config(i2s_config);
  a2d_sink.set_pin_config(pin_config);
  a2d_sink.start("MyMusic");
  Serial.begin(115200);
}

unsigned long last = 0;

void loop() {
  if ((millis() - last) > 100) {
    last = millis();
    Serial.println(a2d_sink.get_audio_state());
    //Serial.println(a2d_sink.get_audio_type());
  }
}

This works for me. I only changed the i2S pins because pins 19, 22 and 23 were already connected to the I2S input of my Teensy. Next I will try to make a simple example program that works with the audio library and at which the T4 receives audio from the ESP32 via I2S. It seems that more people are interested in connecting an ESP32 to the Teensy. So it is probably helpful if there is a combination of a T4 program and an ESP32 that work together.

[SomeoneFromGermany]

You used the same I2S config for the teensy as you mentioned before?

[alex6679]

Yes, the code above is part of the I2S config on the Teensy, that I used. The problem is that it is part of a quite large project. Posting the complete project would not help much, since the project is quite large and not compatible the audio library. But I will make a short example as mentioned above. Today I used the ESP32 as I2S master. I will also try to use the T4 as master. But if remember correctly I ran into a problem the last time I tried that.

[alex6679]

I just committed a first working example of an I2S stream from the ESP32 to the Teensy 4:
https://github.com/alex6679/ESP32_I2.../main/example1

ESP32 side:
I used the esp32_bt_music_receiver from https://github.com/dvxlab/esp32_bt_music_receiver to receive audio data with the ESP32. The ESP32 is I2S master and sends 16bit samples.

Teensy 4:
At the Teensy I basically used AudioInputI2Sslave to receive the data. I only changed the number of received bits from 32 to 16. For that reason I copied input_i2s.h, input_i2s.cpp, output_i2s.h and output_i2s.cpp (config_i2s methode is needed) and extended the names of the file and the classes with _esp32 so that there are no conflicts with the files of the Audio library. The only important lines of code that I changed were the configuration of I2S1_RCR4 and I2S1_RCR5 as already mentioned above. The audio data is send from the I2S input to a plotter class that sent the data via Serial to my notebook. I have to admit that I don't have the possibility to listen to the received audio, but the plots with the Arduino IDE Serial plotter look as if the signal is OK:



Then I tried to switch to 32 bit samples. At the Teensy I only reverted my changes to I2S1_RCR4 and I2S1_RCR5. At the ESP32 I change 'bit_per_sample' to 32:

  static const i2s_config_t i2s_config = {
    .mode = (i2s_mode_t) (I2S_MODE_MASTER | I2S_MODE_TX),
    .sample_rate = 44100,
    .bits_per_sample = (i2s_bits_per_sample_t)32,
    .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
    .communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
    .intr_alloc_flags = 0, // default interrupt priority
    .dma_buf_count = 8,
    .dma_buf_len = 128,//64,
    .use_apll = true,
    .tx_desc_auto_clear = true                                              //Auto clear tx descriptor on underflow
  };

Here is an image of the received waveform with the 32bit configuration:

I suspect that the I2S driver of the ESP32 just doesn't support to transmit 32 bit samples.
Next I will try to get a simple example with the Teensy as I2S master and the ESP32 as slave.

[alex6679]

I think I also got a working example with the Teensy as I2S master: https://github.com/alex6679/ESP32_I2.../main/example2
I only change a single line of code at the ESP32:

  static const i2s_config_t i2s_config = {
    .mode = (i2s_mode_t) (I2S_MODE_MASTER | I2S_MODE_TX),  //only I2S_MODE_MASTER was changed to I2S_MODE_SLAVE
    .sample_rate = 44100,
    .bits_per_sample = (i2s_bits_per_sample_t)16,
    .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
    .communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
    .intr_alloc_flags = 0, // default interrupt priority
    .dma_buf_count = 8,
    .dma_buf_len = 128,//64,
    .use_apll = true,
    .tx_desc_auto_clear = true                                              //Auto clear tx descriptor on underflow
  };

At the Teensy I changed AudioOutputI2S_ESP32::config_i2s(). The complete code can be found in the repository above. The relevant lines are:

    // bit clock divider of the receiver
    I2S1_RCR2 = I2S_RCR2_SYNC(rsync) | I2S_RCR2_BCP  // sync=0; rx is async;
        | (I2S_RCR2_BCD |
	//I2S_RCR2_DIV((1))	//The division value is (DIV + 1) * 2. -> a factor of 4 for 32 bit samples
	I2S_RCR2_DIV((3))	//16 bit samples: the divider is set to 8
	| I2S_RCR2_MSEL(1));

    uint32_t noBits = 16;
    uint32_t noBitsM1=noBits-1;
    I2S1_RCR4 = I2S_RCR4_FRSZ(1)
				| I2S_RCR4_SYWD(noBitsM1)
				| I2S_RCR4_MF
				| I2S_RCR4_FSE
				| I2S_RCR4_FSP
				| I2S_RCR4_FSD;    //Frame sync direction is set to 1 when the Teensy is I2S master
    I2S1_RCR5 = I2S_RCR5_WNW(noBitsM1)
				| I2S_RCR5_W0W(noBitsM1)
				| I2S_RCR5_FBT(31);

The nice thing is that the bitclock can be divided by different factors at the I2S receiver and transmitter. So I think it should be possible to receive 16 bit samples from the ESP32 and send 32 bit samples to the audio shield at the same time.

The received signal:

I marked two positions at the signal that look suspicious. I am not sure if the Teensy or the ESP32 inserted zeros at this positions. Maybe there was a buffer underrun? I will try to output the signal via SPDIF so that I can listen to it over my active speaker.

[alex6679]

Qick update: I had a closer look at the signal in case the Teensy is I2S master. Indeed there is still problem, when the ESP32 is I2S slave

[SomeoneFromGermany]

I tried your example2 code and changed the pins of the I2S interface of the ESP32 to:

#define CONFIG_EXAMPLE_I2S_LRCK_PIN 25
#define CONFIG_EXAMPLE_I2S_BCK_PIN 26
#define CONFIG_EXAMPLE_I2S_DATA_PIN 22

because I already soldered it like this.
I made no changes to the teensys code.

All i receive is this even with audio paused:

On top of this i receive no audio at all from the speaker connected to the audioshield.
I will try soldering to other pins tomorow.