ESP32 as I2S input-device

Status
Not open for further replies.
Maybe your speaker was playing the signal of your screenshot. The signal is just very quite. Anyway, there is a problem. Can you remove the audioshield for now and just concentrate on receiving the correct signal from the ESP32?


Ok, I would like to summerize what I found out:
I had a look at three different possibilities to connect the ESP32 to the T4. At all three options I configured the I2S signal for 16bit samples. I wasn't able to get it working with 32bit samples.

1. The T4 is I2S slave and the ESP32 is master:
Example code can be found here: https://github.com/alex6679/ESP32_I2S_Teensy4/tree/main/example1
This seems to work, but I think there are then many restrictions on what can be done with the audio signal at the Teensy. Using for example the spdif output would probably not work well since it is clocked by the PLL4 of the Teensy.

2. The T4 is I2S master and the ESP32 is slave:
Example code can be found here: https://github.com/alex6679/ESP32_I2S_Teensy4/tree/main/example2
I think I found the correct I2S configuration of the Teensy, but zeros were inserted into the audio signal. I found a thread were people were discussung some problems when the ESP32 is running as I2S slave:
https://www.esp32.com/viewtopic.php?t=16290
It seems that the ESP32 can only work correctly in slave mode, if its internal master clock is in sync with the provided bit and frame clocks. This was obviously not the case at my experiments since the clock at the Teensy is generated by its internal PLL. However, the T4 offers lots of possibilities at the clock routing and it is maybe possible to use the master clock from the ESP32 to clock the I2S interface of Teensy. Then this configuration should work.

3. The T4 runs in slave mode, but the input is resampled:
The files from https://github.com/alex6679/teensy-4-async-inputs are needed and the example is here: https://github.com/alex6679/teensy-4-async-inputs/tree/main/exampleESP32
At this example the audio data is resampled and forwarded to the spdif output. The plots of the wave looked correct and the signal sounded fine over my active speaker. The example works with the first I2S input.
If needed, I would provide an example for the second I2S input/ SAI2. Then the first I2S interface can be used for the audioshield.
 
I dont know what im doing wrong. I receive no data at the teensy with your second solution.
I quadruple-checked my connections and changed the ESP32, but the teensy shows no reaction. Plotter stays on zero.
(I removed the audioshield beforehand)
 
Which pins do you use at the Teensy? With my code it should be IN1, BCLK1, LRCLK1. I am just asking because at one point you used AudioInputI2SQuad and I am wondering if you used connected the second input to the ESP.
What also just came to my mind: Could it be that the ESP32 starts before the Teensy so that there are no clock signals present when the ESP32 starts? Maybe that could cause a problem. At my tests I first connected the Teensy to my notebook and then turned on the ESP32. So the clock signals were always present, when the ESP turned on.
Did you also have a look at the signals with an oscilloscope?
 
Hello alex6679,

Please excuse my lack of knowledge here, but how is this project compiled? Is it a library? The examples have .cpp files which don't open in Arduino, or Platform.io. We would need a ini or ino project file? I have my hardware here and I'm going to give your examples a try, but I'm stuck. As silly as this question may be, any help would be appreciated.

If needed, I would provide an example for the second I2S input/ SAI2. Then the first I2S interface can be used for the audioshield.

This is the golden ticket. It would be great to see an Arduino example that would play the audio received from the bluetooth sink through the audio shield. (ESP32 -> T4 -> Audio Shield). I envision the audio shield on SAI1 and the ESP32 on SAI2. I do believe that both SAI ports can have separate clocks too, which may help.
 
You can rename the "main.cpp" file to "main.ino", but you have to be careful, because arduino IDE wants you to put the file in an folder called main. Make sure that you copy the other .h and .cpp in the same folder.

@alex6679
Im using I2S1 with pin 20, 21 and 8.
I used your code and connections.
I dont have the oszilloscope on hand right now, because i used the one from my holiday-job.

Wait... or not. I'm fixing it later.
 
Last edited:
Please excuse my lack of knowledge here, but how is this project compiled?
Sorry, I completely forgot about that. I use Visual Studio Code + VisualTeensy to compile my projects. That's the reason why I have main.cpp files instead of .ino files. As SomeoneFromGermany suggested: Just change the file extension and place the files in a folder with the name of the .ino file.

Regarding using SAI2 for the ESP32: The only problem I have is that I am not able to test it. I would need to disassemble my surround processor in which the Teensys and the ESP32 are running. Therefore I would need somebody who could help me and test the example with the ESP32 connected to SAI2.
 
Sorry, I completely forgot about that. I use Visual Studio Code + VisualTeensy to compile my projects. That's the reason why I have main.cpp files instead of .ino files. As SomeoneFromGermany suggested: Just change the file extension and place the files in a folder with the name of the .ino file.

No problem. I figured as much, but wasn't sure. I also wasn't aware of VisualTeensy. I tried VT, but wound up just renaming the files and used Arduino. I compiled both ESP32 and Teensy example 1 and I have my wires hooked up. I can pair bluetooth with the ESP32. But nothing shows up in the Serial Plotter. I'm not sure where to go from here, but I'll keep trying.

Regarding using SAI2 for the ESP32: The only problem I have is that I am not able to test it. I would need to disassemble my surround processor in which the Teensys and the ESP32 are running. Therefore I would need somebody who could help me and test the example with the ESP32 connected to SAI2.

FWIW, I'd happily send you money to be used towards a T4, Audioshield, and ESP32 Dev Module if you want. Looks like the T4+AS = 33.70 and an ESP32 is about $11. So $55 should cover it including shipping. I would also be happy to continue running tests on my hardware, just let me know if you want to take me up on the offer (PM?). I appreciate you taking the time to create the examples you've provided thus far.

SomeoneFromGermany said:
With SAI2 you mean I2S2?

Yes. SAI is the Synchronous Audio Interface aka Serial Audio Interface. The Teensy exposes two of them. The second one only has one serial input and one serial output though. I do believe that we can run the Teensy SAI2/I2S2 in slave mode, all while the SAI1/I2S1 continues to run in master mode; possibly avoiding the issue Alex6679 brought up when the Teensy is the master. (Disclaimer: I might be wrong).
 
Since you both have problems with the examples, I will double check tomorrow evening if I messed something up when I committed the files. Although I can't imagine that I committed the wrong files. I use a ESP32 Devkit. I will have a look which version I have. Maybe this is the difference between our setups. I will also check which esp-idf and Teensyduino versions I have installed.
Are you sure that the ESP sends out audio data? It would be good if we could take the bluetooth out of the equation and just generate a sine wave on the ESP. But unfortunately I don't know how to do that. I don't have much experience with the audio stuff of the ESP32. Until now I just compiled the bluetooth examples and they worked out of the box.
Have you tried both examples or only one?

Regarding your offer to send me money for the equipment: Let's first see if we get SAI1 + ESP32 running. I assume that switching to SAI2 and adding the audiosshield won't be a deal then.
 
My hardware.

  • Feather HUZZAH32
  • Teensy 4.1


I modified the pins on the ESP32 to make it easier to find the right pins. These pinouts were easily found so I used them.

feather_gpio.jpg

Bottom row:
A0 - this is an analog input A0 and also an analog output DAC2. It can also be used as a GPIO #26. It uses ADC #2
A1 - this is an analog input A1 and also an analog output DAC1. It can also be used as a GPIO #25. It uses ADC #2
A2 - this is an analog input A2 and also GPI #34. Note it is not an output-capable pin! It uses ADC #1
A3 - this is an analog input A3 and also GPI #39. Note it is not an output-capable pin! It uses ADC #1
A4 - this is an analog input A4 and also GPI #36. Note it is not an output-capable pin! It uses ADC #1
A5 - this is an analog input A5 and also GPIO #4. It uses ADC #2
21 - General purpose IO pin #21
source: https://learn.adafruit.com/adafruit-huzzah32-esp32-feather/pinouts?view=all

Code:
#define CONFIG_EXAMPLE_I2S_BCK_PIN 26 // (A0 on huzzah32, pin 21 on T4.1)
#define CONFIG_EXAMPLE_I2S_LRCK_PIN 25 // (A1 on huzzah32, pin 20 on T4.1)
#define CONFIG_EXAMPLE_I2S_DATA_PIN 4 // (A5 on huzzah32, pin 8 on T4.1)
// T4.1 GND is connected to huzzah32 GND
source: https://www.pjrc.com/store/teensy41.html

The hardware is on a breadboard

20210915_175420 (1).jpg

And I get this in the Plotter. It is counting forward.

Screenshot 2021-09-15 175809.png

If I disconnect the clocks (LRK, BCLK) the plotter stops counting. Does this mean the clocks are working?

If I disconnect the data line nothing happens.

This was running example 1. I'll try the other example, and I'll try some other GPIO pins for the data line to see if I can get any different results.
 
I changed to pin 21 and example 1 works!

Code:
#define CONFIG_EXAMPLE_I2S_BCK_PIN 26 // (A0 on huzzah32, pin 21 on T4.1)
#define CONFIG_EXAMPLE_I2S_LRCK_PIN 25 // (A1 on huzzah32, pin 20 on T4.1)
#define CONFIG_EXAMPLE_I2S_DATA_PIN [B]21 [/B]// (21 on huzzah32, pin 8 on T4.1)
// T4.1 GND is connected to huzzah32 GND



Screenshot 2021-09-15 185937.png
 
I am sure that the ESP32 outputs audio data.
In my test with the 8-bit dacs there was definitively audio. I tested a 1Hz sinewave and with the oszilloscope it was visible that it changed the way I expected (Dont have pictures though).

I also use an ESP32 Devkit.
I soldered everything with headers to a perfboard.
 
I changed to pin 21 and example 1 works!
Great, I am glad that it works now.
I do believe that we can run the Teensy SAI2/I2S2 in slave mode, all while the SAI1/I2S1 continues to run in master mode; possibly avoiding the issue Alex6679 brought up when the Teensy is the master.
I think it is not that simple. Currently SAI1 is clocked bei Teensys PLL4 and if we us SAI2 in slave mode it will be clocked by the ESP32. Since the clocks of the Teensy and the ESP don't have exactly the same frequencies, buffer under- and overflows will appear. I think the simplest solution to that problem is option 3 that I suggest above.
I would love to.
Great, I will prepare examples. However, I think you should first try to get the SAI1 examples working.

Regarding my setup
Teensy: I use a Teensy 4.0 and Teensyduino 1.53 is installed on my Notebook.
ESP32: I have a ESP-DEVKITC-32U. I use PlatformIO with Espressif 32 v3.3.2

@SomeoneFromGermany: Is example1 working? If not: Does the Plotter show zeros or nothing at all?
 
Its working.

Screenshot 2021-09-16 161039.png
There are also insertet zeros.

I played a 12Hz sinewave:
Screenshot 2021-09-16 161301.png
after what I have seen so far there are quite many zeros inserted.

I dont think this will be a problem, but I'm using teensyduino 1.54.

*and if you do:
Code:
if (Serial) {
    Serial.print(j == 0 ? "Left:" : "Right:");
    Serial.print(*dataPtr[j]);
    Serial.print(" ");
}
In your plotter.cpp, you will see what colors each channel has.:D
 
I think it is not that simple. Currently SAI1 is clocked bei Teensys PLL4 and if we us SAI2 in slave mode it will be clocked by the ESP32. Since the clocks of the Teensy and the ESP don't have exactly the same frequencies, buffer under- and overflows will appear. I think the simplest solution to that problem is option 3 that I suggest above.

Ah, you are right. I can see the problem with over/underflows. Your solution makes sense to me.

I want to confirm that example 2 now works for me too. The same hardware setup can be easily used to swap between ESP32 as master and Teensy 4.1 as master. Cool! FWIW, I forked your examples and made them arduino compatible for ease of testing on my hardware. https://github.com/JayShoe/ESP32_I2S_Teensy4

3. The T4 runs in slave mode, but the input is resampled:
The files from https://github.com/alex6679/teensy-4-async-inputs are needed and the example is here: https://github.com/alex6679/teensy-4...n/exampleESP32
At this example the audio data is resampled and forwarded to the spdif output. The plots of the wave looked correct and the signal sounded fine over my active speaker. The example works with the first I2S input.
If needed, I would provide an example for the second I2S input/ SAI2. Then the first I2S interface can be used for the audioshield.

I now am trying example3 but I'm getting errors about the Resampler. I copied (what I thought was) the needed files over to a new "example3" folder and here is what I have so far. https://github.com/JayShoe/ESP32_I2S_Teensy4/tree/main/example3

The errors are as follows.

Code:
In file included from E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\main.ino:6:0:
E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\async_input.h: In member function 'double AsyncAudioInput<TInput>::getAttenuation() const':
async_input.h:129: error: 'const class Resampler' has no member named 'getAttenuation'
   double a= _resampler.getAttenuation();
                        ^
E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\async_input.h: In member function 'int32_t AsyncAudioInput<TInput>::getHalfFilterLength() const':
async_input.h:136: error: 'const class Resampler' has no member named 'getHalfFilterLength'
   int32_t l= _resampler.getHalfFilterLength();
                         ^
E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\async_input.h: At global scope:
async_input.h:297: error: 'AsyncAudioInputI2Sslave' was not declared in this scope
 typedef AsyncAudioInput<AsyncAudioInputI2Sslave> AsyncAudioInputI2S;
                         ^
async_input.h:297: error: template argument 1 is invalid
 typedef AsyncAudioInput<AsyncAudioInputI2Sslave> AsyncAudioInputI2S;
                                                ^
E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\async_input.h: In instantiation of 'AsyncAudioInput<TInput>::AsyncAudioInput(bool, bool, float, int32_t, int32_t) [with TInput = AsyncAudioInputI2Sslave_esp32; int32_t = long int]':
E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\main.ino:22:48:   required from here
async_input.h:54: error: no matching function for call to 'Resampler::Resampler(float&, int32_t&, int32_t&)'
   _resampler(attenuation, minHalfFilterLength, maxHalfFilterLength)
                                                                   ^
In file included from E:\OneDrive\Documents\Arduino\libraries\Audio/async_input_spdif3.h:31:0,
                 from E:\OneDrive\Documents\Arduino\libraries\Audio/Audio.h:68,
                 from E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\main.ino:3:
E:\OneDrive\Documents\Arduino\libraries\Audio/Resampler.h:52:9: note: candidate: Resampler::Resampler(Resampler::StepAdaptionParameters)
         Resampler(StepAdaptionParameters settings=StepAdaptionParameters());
         ^
E:\OneDrive\Documents\Arduino\libraries\Audio/Resampler.h:52:9: note:   candidate expects 1 argument, 3 provided
E:\OneDrive\Documents\Arduino\libraries\Audio/Resampler.h:41:7: note: candidate: constexpr Resampler::Resampler(const Resampler&)
 class Resampler {
       ^
E:\OneDrive\Documents\Arduino\libraries\Audio/Resampler.h:41:7: note:   candidate expects 1 argument, 3 provided
E:\OneDrive\Documents\Arduino\libraries\Audio/Resampler.h:41:7: note: candidate: constexpr Resampler::Resampler(Resampler&&)
E:\OneDrive\Documents\Arduino\libraries\Audio/Resampler.h:41:7: note:   candidate expects 1 argument, 3 provided
Multiple libraries were found for "Audio.h"
 Used: E:\OneDrive\Documents\Arduino\libraries\Audio
 Not used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\Audio
Multiple libraries were found for "SD.h"
 Used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\SD
 Not used: C:\Program Files (x86)\Arduino\libraries\SD
'const class Resampler' has no member named 'getAttenuation'
 
For the ESP32 side of things, if someone wonders, yes the ESP32 can use audio and SerialBt at the same time:

Code:
#include <arduino.h>
#include "esp32_bt_music_receiver.h"
#include "BluetoothSerial.h"

#define CONFIG_EXAMPLE_I2S_LRCK_PIN 23
#define CONFIG_EXAMPLE_I2S_BCK_PIN 22
#define CONFIG_EXAMPLE_I2S_DATA_PIN 19

BlootoothA2DSink a2d_sink;

#if !defined(CONFIG_BT_ENABLED) || !defined(CONFIG_BLUEDROID_ENABLED)
#error Bluetooth is not enabled! Please run `make menuconfig` to and enable it
#else
BluetoothSerial SerialBT;
#endif

void setup() {
  static const i2s_config_t i2s_config = {
    //.mode = (i2s_mode_t) (I2S_MODE_MASTER | I2S_MODE_TX),
    .mode = (i2s_mode_t) (I2S_MODE_SLAVE | I2S_MODE_TX),
    .sample_rate = 44100,
    .bits_per_sample = (i2s_bits_per_sample_t)16,
    .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
    .communication_format = (i2s_comm_format_t) (I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
    .intr_alloc_flags = 0, // default interrupt priority
    .dma_buf_count = 8,
    .dma_buf_len = 128,//64,
    .use_apll = true,
    .tx_desc_auto_clear = true                                              //Auto clear tx descriptor on underflow
  };
  i2s_pin_config_t pin_config = {
    .bck_io_num = CONFIG_EXAMPLE_I2S_BCK_PIN,
    .ws_io_num = CONFIG_EXAMPLE_I2S_LRCK_PIN,
    .data_out_num = CONFIG_EXAMPLE_I2S_DATA_PIN,
    .data_in_num = -1                                                       //Not used
  };

  a2d_sink.set_i2s_config(i2s_config);
  a2d_sink.set_pin_config(pin_config);
  a2d_sink.start("MyMusic");                     //Name won't show because the second set name is shown as Bt audio, but also receives Serial data.
  SerialBT.begin("ESP32test");
  Serial.begin(115200);
}

unsigned long last = 0;

void loop() {
  if ((millis() - last) > 100) {
    last = millis();
    Serial.println(a2d_sink.get_audio_state());
    //Serial.println(a2d_sink.get_audio_type());
  }

  if (Serial.available()) {
    SerialBT.write(Serial.read());
  }
  if (SerialBT.available()) {
    Serial.write(SerialBT.read());
  }
}
 
I am glad that it works now for both of you. What were the problems? Did you have to change something in the example code? I would add all useful information to the documentation of the examples.
In your plotter.cpp, you will see what colors each channel has.
I just added it (not committed yet).

@JayShoe
Do you use an old Teensyduino version? If yes, could you update to 1.53 or higher?

I just started working on the SAI2 interface. Unfortunately I noticed that the implementation of the AudioOutputI2S2slave class is deactivated in the audio library. So there is maybe a problem with that code. But let's see.
 
I only soldered the pin to other pins on the ESP32 and not the teensy... So the DOUT of the ESP32 was on the wrong pin...

So you mean you miss the foundation for the SAI2-slave config?
Maybe Paul knows how this works.

If there would be a problem with teensyduino 1.53 -> 1.54 i could test that, because i have both versions.
But at this point both programs (ESP32 and teensy) compiles and works.
I could also test older versions.

EDIT:
There is definitiley a AudioOutputI2S2slave class with the AudioOutputI2S2slave::config_i2s function, but I dont know if you mean this.
 
Last edited:
I was only asking JoyShoe for the Teensyduino version because of the compiler error that he postet. The compiler is complaining about the interface of the Resampler class. That interface changed some time ago and I wrote the example for the current interface. teensyduino 1.53, 1.54 or 1.55 should all work with the third example.
Yes, I mainly meant AudioOutputI2S2slave::config_i2s. It is there but the whole class is deactivated with a #ifdef 0 ... #endif around it. I think Frank B implemented the class and maybe there was just not enough time to finish the implementation of the slave input class.
I just compared the code with the implementation of the slave input for SAI1 and adopted it accordingly . I hope, I haven't missed anything. But we will see. Tomorrow I will add the small class that enables the resampling and then you could try the three ESP32 examples with SAI2.
 
Teensyduino 1.53, 1.54 or 1.55 should all work with the third example.

I fixed an issue with my version of the Audio library and upgraded to 1.54. Now the error is this.

Code:
Arduino: 1.8.15 (Windows 10), TD: 1.54, Board: "Teensy 4.1, Serial, 600 MHz, Faster, US English"

In file included from E:\Github\ESP32_I2S_Teensy4_forked\example3\teensy4\main\main.ino:6:0:

async_input.h:297: error: 'AsyncAudioInputI2Sslave' was not declared in this scope

 typedef AsyncAudioInput<AsyncAudioInputI2Sslave> AsyncAudioInputI2S;

                         ^

async_input.h:297: error: template argument 1 is invalid

 typedef AsyncAudioInput<AsyncAudioInputI2Sslave> AsyncAudioInputI2S;

                                                ^

Multiple libraries were found for "SD.h"

 Used: C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\SD

 Not used: C:\Program Files (x86)\Arduino\libraries\SD

'AsyncAudioInputI2Sslave' was not declared in this scope



This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.
 
I forgot to commit that I removed that typedef in async_inputs.h. It should work now if you get the latest files from teensy-4-async-inputs.
 
I committed 3 examples for SAI2:
example4: Teensy is slave
example5: Teensy is master
example6: Teensy is slave + resampling

All I can say is that they compile. Let me know if they also work.

Also, I moved all files that are needed at all examples into the top level folder.
 
I got example 3 compiled and running now too.

Example 4 compiles, but only clocks are working on the Plotter.

Example 5 compiles, but the plotter has a broken waveform.

EXAMPLE5.png

Example 6 compiles, but only clocks are working on the Plotter.

Since example 5 shows some sine of working (pun intended), I'm pretty sure that the hardware is correct. An interesting note about examples 4-6; the clocks work whether the LRCK2/BCLK2 is connected at all.

I committed 3 examples for SAI2:
example4: Teensy is slave
example5: Teensy is master
example6: Teensy is slave + resampling

OK, so in your examples are both SAI1 and SAI2 running on the same clocks? Isn't the idea to have teensy's SAI2 in slave mode, and SAI1 in master? Is that what example6 is supposed to show? Teensy SAI2 is slave + resampled to Teensy SAI1 as master?

And thanks again alex6679 for your contribution. I hope my tests are helpful.

My updates have been pushed up to my fork that is configured for arduino (and lazy compiling with all the files duplicated into the folders).
 
Last edited:
Status
Not open for further replies.
Back
Top