Number of simultaneous mics possible

maxnabokow · Sep 2, 2023

Hi there,

I am working on a project for which I need to run multiple microphone inputs simultaneously. Ideal would be about 8. I initially got these MEMS PDM mics, but then realized the Teensy 4.1 only has 2 PDM inputs.
How many I2S mics inputs could I run simultaneously? Would I be able to do even more with analog mics? What would be the optimal setup? I've got a couple Audio Shields available too if that can help me get more. Any advice helps

Thanks!

PaulStoffregen · Sep 3, 2023

maxnabokow said:
How many I2S mics inputs could I run simultaneously?

The short answer is 10. To accomplish this, you would use these 2 audio input features:

https://www.pjrc.com/teensy/gui/?info=AudioInputI2SOct

https://www.pjrc.com/teensy/gui/?info=AudioInputI2S2

Refer to the right side panel documentation for details.

Each I2S data input pin can have 2 channels. One mic must has its SEL pin wired high and the other wired low.

Since you linked to Adafruit's SPH0645 board, in Arduino, click File > Examples > Audio > HardwareTesting > Microphones > SPH0645. You'll see it links to this thread where use of this particular I2S mic was discussed.

https://forum.pjrc.com/threads/60599?p=238070&viewfull=1#post238070

Scroll down to the info about this mic's DC offset and what you can do if that matters.

maxnabokow · Sep 3, 2023

Wonderful, thank you Paul. Will get some of those I2S mics ordered.

While I wait for them to deliver, is 2 really the maximum number of PDM inputs I can read? I've even got another Teensy 4.1, which should at least get me to 4. Could Audio Shields get me more?

PaulStoffregen · Sep 3, 2023

With the software that exists today, indeed 2 is the max for PDM.

Perhaps FlexIO could be used to get more? But getting that working is no easy task.

maxnabokow · Oct 9, 2023

Hi @Paul,

Thanks for your advice here. I've built my mic array with eight SPH0645 I2S microphones (photo of my setup) and am able to record from them all simultaneously, but I'm running into some issues. Code is at the bottom of this post.

1. The sound quality is atrocious (here are the .wav files). It's cracking horribly and is overall much worse than when I was using my two PDM mics. I haven't figured out yet how to effectively write the .wav header myself in code, so I'm using the Switch tool to convert .RAW to .wav for now (here's my encoder config). I've done some filtering/amping like you recommended, but it's still horrible.
2. Somehow, the first 2 generated .RAW files are always much smaller in size than the other 6. After converting to .wav, that results in significantly sped-up sound (see mic_00.wav and mic_01.wav, vs. the others). I'm not doing anything differently for those in the code and the wiring is the same, so I can't figure out what could cause this. I obviously want all files to be the exact same size.

Code:

[COLOR=#4E5B61][FONT=Menlo][COLOR=#728e00]#include[/COLOR] [COLOR=#005c5f]<Bounce.h>[/COLOR]
[COLOR=#728e00]#include[/COLOR] [COLOR=#005c5f]<Audio.h>[/COLOR]
[COLOR=#728e00]#include[/COLOR] [COLOR=#005c5f]<Wire.h>[/COLOR]
[COLOR=#728e00]#include[/COLOR] [COLOR=#005c5f]<SPI.h>[/COLOR]
[COLOR=#728e00]#include[/COLOR] [COLOR=#005c5f]<SD.h>[/COLOR]
[COLOR=#728e00]#include[/COLOR] [COLOR=#005c5f]<SerialFlash.h>[/COLOR]

[COLOR=#00979d]const[/COLOR] [COLOR=#00979d]int[/COLOR] NUM_CHANNELS = [COLOR=#005c5f]8[/COLOR];

AudioInputI2SOct i2s_oct1;
AudioInputI2S2 i2s2_1;

AudioFilterStateVariable [COLOR=#d35400]filters[/COLOR][NUM_CHANNELS];
AudioAmplifier           [COLOR=#d35400]amps[/COLOR][NUM_CHANNELS];
AudioRecordQueue         [COLOR=#d35400]queues[/COLOR][NUM_CHANNELS];

AudioConnection          [COLOR=#d35400]patchCord1[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]0[/COLOR], filters[[COLOR=#005c5f]0[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord2[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]1[/COLOR], filters[[COLOR=#005c5f]1[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord3[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]2[/COLOR], filters[[COLOR=#005c5f]2[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord4[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]3[/COLOR], filters[[COLOR=#005c5f]3[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord5[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]4[/COLOR], filters[[COLOR=#005c5f]4[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord6[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]5[/COLOR], filters[[COLOR=#005c5f]5[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord7[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]6[/COLOR], filters[[COLOR=#005c5f]6[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord8[/COLOR][COLOR=#434f54]([/COLOR]i2s_oct1, [COLOR=#005c5f]7[/COLOR], filters[[COLOR=#005c5f]7[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
[COLOR=#95a5a6]// AudioConnection          patchCord9(i2s2_1, 0, filters[8], 0);[/COLOR]
[COLOR=#95a5a6]// AudioConnection          patchCord10(i2s2_1, 1, filters[9], 0);[/COLOR]
AudioConnection          [COLOR=#d35400]patchCord11[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]0[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]0[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord12[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]1[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]1[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord13[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]2[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]2[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord14[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]3[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]3[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord15[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]4[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]4[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord16[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]5[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]5[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord17[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]6[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]6[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord18[/COLOR][COLOR=#434f54]([/COLOR]filters[[COLOR=#005c5f]7[/COLOR]], [COLOR=#005c5f]2[/COLOR], amps[[COLOR=#005c5f]7[/COLOR]], [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR];
[COLOR=#95a5a6]// AudioConnection          patchCord19(filters[8], 2, amps[8], 0);[/COLOR]
[COLOR=#95a5a6]// AudioConnection          patchCord20(filters[9], 2, amps[9], 0);[/COLOR]
AudioConnection          [COLOR=#d35400]patchCord21[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]0[/COLOR]], queues[[COLOR=#005c5f]0[/COLOR]][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord22[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]1[/COLOR]], queues[[COLOR=#005c5f]1[/COLOR]][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord23[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]2[/COLOR]], queues[[COLOR=#005c5f]2[/COLOR]][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord24[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]3[/COLOR]], queues[[COLOR=#005c5f]3[/COLOR]][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord25[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]4[/COLOR]], queues[[COLOR=#005c5f]4[/COLOR]][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord26[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]5[/COLOR]], queues[[COLOR=#005c5f]5[/COLOR]][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord27[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]6[/COLOR]], queues[[COLOR=#005c5f]6[/COLOR]][COLOR=#434f54])[/COLOR];
AudioConnection          [COLOR=#d35400]patchCord28[/COLOR][COLOR=#434f54]([/COLOR]amps[[COLOR=#005c5f]7[/COLOR]], queues[[COLOR=#005c5f]7[/COLOR]][COLOR=#434f54])[/COLOR];
[COLOR=#95a5a6]// AudioConnection          patchCord29(amps[8], queues[8]);[/COLOR]
[COLOR=#95a5a6]// AudioConnection          patchCord30(amps[9], queues[9]);[/COLOR]

[COLOR=#728e00]#define[/COLOR] [COLOR=#d35400]SDCARD_CS_PIN[/COLOR] BUILTIN_SDCARD

[COLOR=#00979d]int[/COLOR] mode = [COLOR=#005c5f]0[/COLOR];[COLOR=#95a5a6]  // 0=stopped, 1=recording, 2=playing[/COLOR]

[COLOR=#95a5a6]// Bounce objects to easily and reliably read the buttons[/COLOR]
Bounce buttonRecord = [COLOR=#d35400]Bounce[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]0[/COLOR], [COLOR=#005c5f]8[/COLOR][COLOR=#434f54])[/COLOR];
Bounce buttonStop = [COLOR=#d35400]Bounce[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]1[/COLOR], [COLOR=#005c5f]8[/COLOR][COLOR=#434f54])[/COLOR];[COLOR=#95a5a6]  // 8 = 8 ms debounce time[/COLOR]

File [COLOR=#d35400]files[/COLOR][NUM_CHANNELS];

[COLOR=#00979d]void[/COLOR] [COLOR=#d35400]setup[/COLOR][COLOR=#434f54]()[/COLOR] [COLOR=#434f54]{[/COLOR]
  [COLOR=#d35400]AudioMemory[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]90[/COLOR][COLOR=#434f54])[/COLOR];
  [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]filters[/COLOR][i].[COLOR=#d35400]frequency[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]60[/COLOR][COLOR=#434f54])[/COLOR];[COLOR=#95a5a6] // filter out DC & extremely low frequencies[/COLOR]
    [COLOR=#d35400]amps[/COLOR][i].[COLOR=#d35400]gain[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]0.5[/COLOR][COLOR=#434f54])[/COLOR];[COLOR=#95a5a6] // make it less loud[/COLOR]
  [COLOR=#434f54]}[/COLOR]

  [COLOR=#d35400]pinMode[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]0[/COLOR], INPUT_PULLUP[COLOR=#434f54])[/COLOR];
  [COLOR=#d35400]pinMode[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]1[/COLOR], INPUT_PULLUP[COLOR=#434f54])[/COLOR];

  [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR]![COLOR=#434f54]([/COLOR][COLOR=#d35400]SD[/COLOR].[COLOR=#d35400]begin[/COLOR][COLOR=#434f54]([/COLOR]SDCARD_CS_PIN[COLOR=#434f54])))[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#728e00]while[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#005c5f]1[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
      [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"❌ Unable to access the SD card"[/COLOR][COLOR=#434f54])[/COLOR];
      [COLOR=#d35400]delay[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]500[/COLOR][COLOR=#434f54])[/COLOR];
    [COLOR=#434f54]}[/COLOR]
  [COLOR=#434f54]}[/COLOR] [COLOR=#728e00]else[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"SD Card found."[/COLOR][COLOR=#434f54])[/COLOR];
  [COLOR=#434f54]}[/COLOR]

  [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"Setup complete."[/COLOR][COLOR=#434f54])[/COLOR];
[COLOR=#434f54]}[/COLOR]

[COLOR=#00979d]void[/COLOR] [COLOR=#d35400]loop[/COLOR][COLOR=#434f54]()[/COLOR] [COLOR=#434f54]{[/COLOR]
  [COLOR=#d35400]buttonRecord[/COLOR].[COLOR=#d35400]update[/COLOR][COLOR=#434f54]()[/COLOR];
  [COLOR=#d35400]buttonStop[/COLOR].[COLOR=#d35400]update[/COLOR][COLOR=#434f54]()[/COLOR];

  [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#d35400]buttonRecord[/COLOR].[COLOR=#d35400]fallingEdge[/COLOR][COLOR=#434f54]())[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"record button pressed"[/COLOR][COLOR=#434f54])[/COLOR];
    [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR]mode == [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#d35400]startRecording[/COLOR][COLOR=#434f54]()[/COLOR];
  [COLOR=#434f54]}[/COLOR]
  [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#d35400]buttonStop[/COLOR].[COLOR=#d35400]fallingEdge[/COLOR][COLOR=#434f54]())[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"stop button pressed"[/COLOR][COLOR=#434f54])[/COLOR];
    [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR]mode == [COLOR=#005c5f]1[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#d35400]stopRecording[/COLOR][COLOR=#434f54]()[/COLOR];
  [COLOR=#434f54]}[/COLOR]

  [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR]mode == [COLOR=#005c5f]1[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]continueRecording[/COLOR][COLOR=#434f54]()[/COLOR];
  [COLOR=#434f54]}[/COLOR]
[COLOR=#434f54]}[/COLOR]

[COLOR=#00979d]void[/COLOR] [COLOR=#d35400]startRecording[/COLOR][COLOR=#434f54]()[/COLOR] [COLOR=#434f54]{[/COLOR]
  [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"📣 startRecording"[/COLOR][COLOR=#434f54])[/COLOR];

  [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    String fileString = [COLOR=#005c5f]"RECORD"[/COLOR] + [COLOR=#d35400]String[/COLOR][COLOR=#434f54]([/COLOR]i[COLOR=#434f54])[/COLOR] + [COLOR=#005c5f]".RAW"[/COLOR];
    [COLOR=#00979d]char[/COLOR]* filePath = [COLOR=#d35400]fileString[/COLOR].[COLOR=#d35400]c_str[/COLOR][COLOR=#434f54]()[/COLOR];

    [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#d35400]SD[/COLOR].[COLOR=#d35400]exists[/COLOR][COLOR=#434f54]([/COLOR]filePath[COLOR=#434f54]))[/COLOR] [COLOR=#434f54]{[/COLOR]
      [COLOR=#d35400]SD[/COLOR].[COLOR=#d35400]remove[/COLOR][COLOR=#434f54]([/COLOR]filePath[COLOR=#434f54])[/COLOR];
    [COLOR=#434f54]}[/COLOR]

    [COLOR=#d35400]files[/COLOR][i] = [COLOR=#d35400]SD[/COLOR].[COLOR=#d35400]open[/COLOR][COLOR=#434f54]([/COLOR]filePath, FILE_WRITE[COLOR=#434f54])[/COLOR];
    [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR]![COLOR=#d35400]files[/COLOR][i][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
      [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"❌ File creation failed: "[/COLOR] + [COLOR=#d35400]String[/COLOR][COLOR=#434f54]([/COLOR]i[COLOR=#434f54]))[/COLOR];
    [COLOR=#434f54]}[/COLOR]
  [COLOR=#434f54]}[/COLOR]

  [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]begin[/COLOR][COLOR=#434f54]()[/COLOR];
[COLOR=#95a5a6]    // Serial.println("🎙️ NOW RECORDING: " + String(i));[/COLOR]
  [COLOR=#434f54]}[/COLOR]

  mode = [COLOR=#005c5f]1[/COLOR];
[COLOR=#434f54]}[/COLOR]

[COLOR=#00979d]void[/COLOR] [COLOR=#d35400]continueRecording[/COLOR][COLOR=#434f54]()[/COLOR] [COLOR=#434f54]{[/COLOR]
  [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]available[/COLOR][COLOR=#434f54]()[/COLOR] >= [COLOR=#005c5f]2[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
      byte [COLOR=#d35400]buffer[/COLOR][[COLOR=#005c5f]512[/COLOR]];
[COLOR=#95a5a6]      // Fetch 2 blocks from the audio library and copy[/COLOR]
[COLOR=#95a5a6]      // into a 512 byte buffer.  The Arduino SD library[/COLOR]
[COLOR=#95a5a6]      // is most efficient when full 512 byte sector size[/COLOR]
[COLOR=#95a5a6]      // writes are used.[/COLOR]
      [COLOR=#d35400]memcpy[/COLOR][COLOR=#434f54]([/COLOR]buffer, [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]readBuffer[/COLOR][COLOR=#434f54]()[/COLOR], [COLOR=#005c5f]256[/COLOR][COLOR=#434f54])[/COLOR];
      [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]freeBuffer[/COLOR][COLOR=#434f54]()[/COLOR];
      [COLOR=#d35400]memcpy[/COLOR][COLOR=#434f54]([/COLOR]buffer + [COLOR=#005c5f]256[/COLOR], [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]readBuffer[/COLOR][COLOR=#434f54]()[/COLOR], [COLOR=#005c5f]256[/COLOR][COLOR=#434f54])[/COLOR];
      [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]freeBuffer[/COLOR][COLOR=#434f54]()[/COLOR];
[COLOR=#95a5a6]      // write all 512 bytes to the SD card[/COLOR]
[COLOR=#95a5a6]      // elapsedMicros usec = 0;[/COLOR]
      [COLOR=#d35400]files[/COLOR][i].[COLOR=#d35400]write[/COLOR][COLOR=#434f54]([/COLOR]buffer, [COLOR=#005c5f]512[/COLOR][COLOR=#434f54])[/COLOR];
[COLOR=#95a5a6]      // Uncomment these lines to see how long SD writes[/COLOR]
[COLOR=#95a5a6]      // are taking.  A pair of audio blocks arrives every[/COLOR]
[COLOR=#95a5a6]      // 5802 microseconds, so hopefully most of the writes[/COLOR]
[COLOR=#95a5a6]      // take well under 5802 us.  Some will take more, as[/COLOR]
[COLOR=#95a5a6]      // the SD library also must write to the FAT tables[/COLOR]
[COLOR=#95a5a6]      // and the SD card controller manages media erase and[/COLOR]
[COLOR=#95a5a6]      // wear leveling.  The queue1 object can buffer[/COLOR]
[COLOR=#95a5a6]      // approximately 301700 us of audio, to allow time[/COLOR]
[COLOR=#95a5a6]      // for occasional high SD card latency, as long as[/COLOR]
[COLOR=#95a5a6]      // the average write time is under 5802 us.[/COLOR]
[COLOR=#95a5a6]      // Serial.print("SD write, us=");[/COLOR]
[COLOR=#95a5a6]      // Serial.println(usec);[/COLOR]
    [COLOR=#434f54]}[/COLOR]
  [COLOR=#434f54]}[/COLOR]
[COLOR=#434f54]}[/COLOR]

[COLOR=#00979d]void[/COLOR] [COLOR=#d35400]stopRecording[/COLOR][COLOR=#434f54]()[/COLOR] [COLOR=#434f54]{[/COLOR]
  [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"📣 stopRecording"[/COLOR][COLOR=#434f54])[/COLOR];

  [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]end[/COLOR][COLOR=#434f54]()[/COLOR];
  [COLOR=#434f54]}[/COLOR]

  [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"✅ ALL QUEUES STOPPED"[/COLOR][COLOR=#434f54])[/COLOR];

  [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR]mode == [COLOR=#005c5f]1[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]

    [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
[COLOR=#95a5a6]      // Serial.print("writing queue: " + String(i));[/COLOR]
      [COLOR=#728e00]while[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]available[/COLOR][COLOR=#434f54]()[/COLOR] > [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
[COLOR=#95a5a6]        // Serial.print(".");[/COLOR]
        [COLOR=#d35400]files[/COLOR][i].[COLOR=#d35400]write[/COLOR][COLOR=#434f54](([/COLOR]byte*[COLOR=#434f54])[/COLOR][COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]readBuffer[/COLOR][COLOR=#434f54]()[/COLOR], [COLOR=#005c5f]256[/COLOR][COLOR=#434f54])[/COLOR];
        [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]freeBuffer[/COLOR][COLOR=#434f54]()[/COLOR];
      [COLOR=#434f54]}[/COLOR]
[COLOR=#95a5a6]      // Serial.println();[/COLOR]
    [COLOR=#434f54]}[/COLOR]

    [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
      [COLOR=#d35400]files[/COLOR][i].[COLOR=#d35400]close[/COLOR][COLOR=#434f54]()[/COLOR];
    [COLOR=#434f54]}[/COLOR]

    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"✅ ALL FILES WRITTEN"[/COLOR][COLOR=#434f54])[/COLOR];
  [COLOR=#434f54]}[/COLOR]

  mode = [COLOR=#005c5f]0[/COLOR];
[COLOR=#434f54]}[/COLOR]
[/FONT][/COLOR]

maxnabokow · Oct 17, 2023

Hi,

I'm following up here with new findings in hopes that someone can help.

With <6 channels: Everything works totally fine. All the .RAWs are the same size, and convert to good-sounding .wav files.
With 6 channels: First signs of problems. All .RAWs still the same size, nice. But after conversion to .wav, I can hear some light cracking sounds and parts of the audio seem to be sped up.
With 7+ channels: Chaos. .RAW file sizes no longer matching. After conversion, strong distortion, sped-up sections, etc.

The fact that this works wonderfully with five or less microphones makes me think that this could have something to do with there being five I2S inputs on the Teensy 4.1, just expanded to ten through L/R separation. But even that doesn't seem fully reasonable, because my first five channels are also L/R separated, going to 2.5 input pins (see wiring setup). So the L/R separation seems to work totally fine?

Any ideas why this starts giving me trouble beyond five mics? Am I wiring something incorrectly? Some clock issue maybe? Or am I missing something in the code?
Let me know if you need any more data/recordings to effectively investigate and I can provide those. Thanks in advance!

Max

maxnabokow · Oct 26, 2023

Hi all,

Still in need of help here. What further info can I provide to help you help me? What questions do you need answered? Any advice helps.

Max

h4yn0nnym0u5e · Oct 26, 2023

A few things:

use the commented-out code to look at how long the SD writes are taking
increase the amount of AudioMemory
write in bigger chunks than just one SD card sector

As-is you'll probably be writing 2 audio blocks to each of your 8 files, when the writes trigger, so each write needs to take less than 2*2.9ms/8, or 0.725ms, on average. That's probably optimistic.

If writes do take a long time, even occasionally, you only have about 11 blocks of AudioMemory per channel, so about 32ms of slack at best. Depending on the worst-case SD write time you find out, you'll likely have to at least triple that.

There's a thread somewhere here looking at SD read and write performance with various write chunk sizes and numbers of files. I had a crack at reproducing the results, and for 8 simultaneous files you really need to write 4kB at a time. You probably need to find that thread and run the code on your setup to find the limits. I had a few SD cards which were woefully poor, and even decent SanDisk cards dropped the overall throughput from 22MB/s to 4MB/s when writing 8 files at once. But that's with 4kB chunks ... they're much worse with 512B chunks. 4MB/s is fine, you only need just over 700kB/s.

maxnabokow · Oct 26, 2023

@h4yn0nnym0u5e, thanks for that advice.

I was getting small glitches even with 5 microphones, and these fixed themselves when I increased the AudioMemory value. Turns out AudioMemoryMax was giving me as much as 799 at times. So, I'm now passing 1600 (instead of 90!) and the glitches on 5 microphones are gone.

However, this seems to have no effect when I'm using >5 microphones. AudioMemoryMemoryMax gives me 896 (every time, for some reason), no matter if I'm using 6 mics or 7, and both sound just as awful as before.

Can you explain more what you mean by writing in bigger chunks?
See code below. I've added a `chunkFactor` for myself to play with in the code as a simple multiplier. When I increase it to 2 (should be bigger chunks then, right?), the sound is still just as awful. Memory usage is also still at 896 every time.

Code:

[COLOR=#4E5B61][FONT=Menlo][COLOR=#00979d]void[/COLOR] [COLOR=#d35400]continueRecording[/COLOR][COLOR=#434f54]()[/COLOR] [COLOR=#434f54]{[/COLOR]
  [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]available[/COLOR][COLOR=#434f54]()[/COLOR] >= [COLOR=#005c5f]2[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
      byte [COLOR=#d35400]buffer[/COLOR][chunkFactor*[COLOR=#005c5f]512[/COLOR]];
[COLOR=#95a5a6]      // Fetch 2 blocks from the audio library and copy[/COLOR]
[COLOR=#95a5a6]      // into a 512 byte buffer.  The Arduino SD library[/COLOR]
[COLOR=#95a5a6]      // is most efficient when full 512 byte sector size[/COLOR]
[COLOR=#95a5a6]      // writes are used.[/COLOR]
      [COLOR=#d35400]memcpy[/COLOR][COLOR=#434f54]([/COLOR]buffer, [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]readBuffer[/COLOR][COLOR=#434f54]()[/COLOR], chunkFactor*[COLOR=#005c5f]256[/COLOR][COLOR=#434f54])[/COLOR];
      [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]freeBuffer[/COLOR][COLOR=#434f54]()[/COLOR];
      [COLOR=#d35400]memcpy[/COLOR][COLOR=#434f54]([/COLOR]buffer + chunkFactor*[COLOR=#005c5f]256[/COLOR], [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]readBuffer[/COLOR][COLOR=#434f54]()[/COLOR], chunkFactor*[COLOR=#005c5f]256[/COLOR][COLOR=#434f54])[/COLOR];
      [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]freeBuffer[/COLOR][COLOR=#434f54]()[/COLOR];
[COLOR=#95a5a6]      // write all 512 bytes to the SD card[/COLOR]
[COLOR=#95a5a6]      // elapsedMicros usec = 0;[/COLOR]
      [COLOR=#d35400]files[/COLOR][i].[COLOR=#d35400]write[/COLOR][COLOR=#434f54]([/COLOR]buffer, chunkFactor*[COLOR=#005c5f]512[/COLOR][COLOR=#434f54])[/COLOR];
[COLOR=#95a5a6]      // Uncomment these lines to see how long SD writes[/COLOR]
[COLOR=#95a5a6]      // are taking.  A pair of audio blocks arrives every[/COLOR]
[COLOR=#95a5a6]      // 5802 microseconds, so hopefully most of the writes[/COLOR]
[COLOR=#95a5a6]      // take well under 5802 us.  Some will take more, as[/COLOR]
[COLOR=#95a5a6]      // the SD library also must write to the FAT tables[/COLOR]
[COLOR=#95a5a6]      // and the SD card controller manages media erase and[/COLOR]
[COLOR=#95a5a6]      // wear leveling.  The queue1 object can buffer[/COLOR]
[COLOR=#95a5a6]      // approximately 301700 us of audio, to allow time[/COLOR]
[COLOR=#95a5a6]      // for occasional high SD card latency, as long as[/COLOR]
[COLOR=#95a5a6]      // the average write time is under 5802 us.[/COLOR]
[COLOR=#95a5a6]      // if (usec > 5802) {[/COLOR]
[COLOR=#95a5a6]      //   Serial.println();[/COLOR]
[COLOR=#95a5a6]      //   Serial.print("SD write, us=");[/COLOR]
[COLOR=#95a5a6]      //   Serial.println(usec);[/COLOR]
[COLOR=#95a5a6]      // } else {[/COLOR]
[COLOR=#95a5a6]      //   Serial.print(".");[/COLOR]
[COLOR=#95a5a6]      // }[/COLOR]
    [COLOR=#434f54]}[/COLOR]
  [COLOR=#434f54]}[/COLOR]
[COLOR=#434f54]}[/COLOR]

[COLOR=#00979d]void[/COLOR] [COLOR=#d35400]stopRecording[/COLOR][COLOR=#434f54]()[/COLOR] [COLOR=#434f54]{[/COLOR]
  [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"&#55357;&#56547; stopRecording"[/COLOR][COLOR=#434f54])[/COLOR];

  [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
    [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]end[/COLOR][COLOR=#434f54]()[/COLOR];
  [COLOR=#434f54]}[/COLOR]

  [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"✅ ALL QUEUES STOPPED"[/COLOR][COLOR=#434f54])[/COLOR];

  [COLOR=#728e00]if[/COLOR] [COLOR=#434f54]([/COLOR]mode == [COLOR=#005c5f]1[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]

    [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
[COLOR=#95a5a6]      // Serial.print("writing queue: " + String(i));[/COLOR]
      [COLOR=#728e00]while[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]available[/COLOR][COLOR=#434f54]()[/COLOR] > [COLOR=#005c5f]0[/COLOR][COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
[COLOR=#95a5a6]        // Serial.print(".");[/COLOR]
        [COLOR=#d35400]files[/COLOR][i].[COLOR=#d35400]write[/COLOR][COLOR=#434f54](([/COLOR]byte*[COLOR=#434f54])[/COLOR][COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]readBuffer[/COLOR][COLOR=#434f54]()[/COLOR], chunkFactor*[COLOR=#005c5f]256[/COLOR][COLOR=#434f54])[/COLOR];
        [COLOR=#d35400]queues[/COLOR][i].[COLOR=#d35400]freeBuffer[/COLOR][COLOR=#434f54]()[/COLOR];
      [COLOR=#434f54]}[/COLOR]
[COLOR=#95a5a6]      // Serial.println();[/COLOR]
    [COLOR=#434f54]}[/COLOR]

    [COLOR=#728e00]for[/COLOR] [COLOR=#434f54]([/COLOR][COLOR=#00979d]int[/COLOR] i = [COLOR=#005c5f]0[/COLOR]; i < NUM_CHANNELS; i++[COLOR=#434f54])[/COLOR] [COLOR=#434f54]{[/COLOR]
      [COLOR=#d35400]files[/COLOR][i].[COLOR=#d35400]close[/COLOR][COLOR=#434f54]()[/COLOR];
    [COLOR=#434f54]}[/COLOR]

    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"✅ ALL FILES WRITTEN"[/COLOR][COLOR=#434f54])[/COLOR];
    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]print[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]"Max memory used: "[/COLOR][COLOR=#434f54])[/COLOR];
    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]print[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#d35400]AudioMemoryUsageMax[/COLOR][COLOR=#434f54]())[/COLOR];
    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]print[/COLOR][COLOR=#434f54]([/COLOR][COLOR=#005c5f]" / "[/COLOR][COLOR=#434f54])[/COLOR];
    [COLOR=#d35400]Serial[/COLOR].[COLOR=#d35400]println[/COLOR][COLOR=#434f54]([/COLOR]audioMemory[COLOR=#434f54])[/COLOR];
  [COLOR=#434f54]}[/COLOR]

  mode = [COLOR=#005c5f]0[/COLOR];
[COLOR=#434f54]}[/COLOR]
[/FONT][/COLOR]

h4yn0nnym0u5e · Oct 27, 2023

Documentation for AudioMemoryUsageMax() is at https://www.pjrc.com/teensy/td_libs_AudioConnection.html … give it a read, I’m sure you’ll figure out why you only ever get the one value. What it doesn’t tell you is that the upper limit for the number of blocks is restricted to 896, so your code is using all of them. That’s enough for a 324ms queue for each mic, when you get to 8 of them, so should be adequate.

You’re about half way to writing bigger chunks - you have the buffer allocated, and write its contents to the file, but you still only check for two blocks available() in each queue, and only copy two to the buffer. I’d expect that to sound worse than before!

maxnabokow · Oct 27, 2023

Update: This worked, thank you! Good to know it was just a memory/SD issue the whole time.

You’re about half way to writing bigger chunks - you have the buffer allocated, and write its contents to the file, but you still only check for two blocks available() in each queue, and only copy two to the buffer.

^ This was the kicker. I'm now using a bigger buffer and filling it with multiple chunks using my `chunkFactor`. Code below. Worked well for all 8 mics with a chunkFactor of 8. Thanks again @h4yn0nnym0u5e.

Code:

[FONT=Verdana]void continueRecording() {
  for (int i = 0; i < NUM_CHANNELS; i++) {[/FONT]
[FONT=Verdana]    if (queues[i].available() >= 8) {[/FONT]
[FONT=Verdana]      byte buffer[chunkFactor*256];[/FONT]
[FONT=Verdana]      // Fetch blocks from the audio library and copy[/FONT]
[FONT=Verdana]      // into a buffer.[/FONT]
[FONT=Verdana]      for (int j = 0; j < chunkFactor; j++) {[/FONT]
[FONT=Verdana]        int offset = j*256;[/FONT]
[FONT=Verdana]        memcpy(buffer + offset, queues[i].readBuffer(), 256);[/FONT]
[FONT=Verdana]        queues[i].freeBuffer();[/FONT]
[FONT=Verdana]      }[/FONT]
[FONT=Verdana]      // write the buffer to the SD card[/FONT]
[FONT=Verdana]      // elapsedMicros usec = 0;[/FONT]
[FONT=Verdana]      files[i].write(buffer, chunkFactor*256);[/FONT]
[FONT=Verdana]      // Uncomment these lines to see how long SD writes[/FONT]
[FONT=Verdana]      // are taking.  A pair of audio blocks arrives every[/FONT]
[FONT=Verdana]      // 5802 microseconds, so hopefully most of the writes[/FONT]
[FONT=Verdana]      // take well under 5802 us.  Some will take more, as[/FONT]
[FONT=Verdana]      // the SD library also must write to the FAT tables[/FONT]
[FONT=Verdana]      // and the SD card controller manages media erase and[/FONT]
[FONT=Verdana]      // wear leveling.  The queue1 object can buffer[/FONT]
[FONT=Verdana]      // approximately 301700 us of audio, to allow time[/FONT]
[FONT=Verdana]      // for occasional high SD card latency, as long as[/FONT]
[FONT=Verdana]      // the average write time is under 5802 us.[/FONT]
[FONT=Verdana]      // if (usec > 5802) {[/FONT]
[FONT=Verdana]      //   Serial.println();[/FONT]
[FONT=Verdana]      //   Serial.print("SD write, us=");[/FONT]
[FONT=Verdana]      //   Serial.println(usec);[/FONT]
[FONT=Verdana]      // } else {[/FONT]
[FONT=Verdana]      //   Serial.print(".");[/FONT]
[FONT=Verdana]      // }[/FONT]
[FONT=Verdana]    }[/FONT]
[FONT=Verdana]  }[/FONT]
[FONT=Verdana]}[/FONT]


[FONT=Verdana]void stopRecording() {[/FONT]
[FONT=Verdana]  Serial.println("📣 stopRecording");[/FONT]


[FONT=Verdana]  for (int i = 0; i < NUM_CHANNELS; i++) {[/FONT]
[FONT=Verdana]    queues[i].end();[/FONT]
[FONT=Verdana]  }[/FONT]


[FONT=Verdana]  Serial.println("✅ ALL QUEUES STOPPED");[/FONT]


[FONT=Verdana]  if (mode == 1) {[/FONT]


[FONT=Verdana]    for (int i = 0; i < NUM_CHANNELS; i++) {[/FONT]
[FONT=Verdana]      // Serial.print("writing queue: " + String(i));[/FONT]
[FONT=Verdana]      while (queues[i].available() > 0) {[/FONT]
[FONT=Verdana]        // Serial.print(".");[/FONT]
[FONT=Verdana]        files[i].write((byte*)queues[i].readBuffer(), 256);[/FONT]
[FONT=Verdana]        queues[i].freeBuffer();[/FONT]
[FONT=Verdana]      }[/FONT]
[FONT=Verdana]      // Serial.println();[/FONT]
[FONT=Verdana]    }[/FONT]


[FONT=Verdana]    for (int i = 0; i < NUM_CHANNELS; i++) {[/FONT]
[FONT=Verdana]      files[i].close();[/FONT]
[FONT=Verdana]    }[/FONT]


[FONT=Verdana]    Serial.println("✅ ALL FILES WRITTEN");[/FONT]
[FONT=Verdana]    Serial.print("Max memory used: ");[/FONT]
[FONT=Verdana]    Serial.print(AudioMemoryUsageMax());[/FONT]
[FONT=Verdana]    Serial.print(" / ");[/FONT]
[FONT=Verdana]    Serial.println(audioMemory);[/FONT]
[FONT=Verdana]  }[/FONT]


[FONT=Verdana]  mode = 0;[/FONT]
[FONT=Verdana]}[/FONT]

h4yn0nnym0u5e · Oct 27, 2023

Excellent news! One point which could become an issue in the future - your queues.available() check should be >=chunkFactor, not >=8. Just out of interest, how many blocks does AudioMemoryUsageMax() report now?

I'm glad this is working, because I've recommended a Teensy 4.1 + 8x I2S mics for a project at work. We're only planning to report timings, not record the audio, but it's good to know we'd have the option if needed.

maxnabokow · Oct 27, 2023

Just out of interest, how many blocks does AudioMemoryUsageMax() report now?

AudioMemoryUsageMax() gives me around 150 with 8 mics and a `chunkFactor` of 8, on a 30-second recording.

h4yn0nnym0u5e · Oct 27, 2023

OK, so you’re aiming to write 8 blocks at at time, which occurs roughly every 23ms, but a max usage of 150 suggests you occasionally get a backlog of 150/8=19 blocks, so sometimes the writes take about 54ms. But not often enough to matter, because overall you’re recording clean audio. Sounds about right to me.

maxnabokow · Dec 14, 2023

Hi @h4yn0nnym0u5e,

I've got a follow-up on this that I'm wondering if you could help me with. Got a research deadline coming up here so need some urgent help...

Rather than saving the mic data to an SD card, I'm now sending it over Serial to a laptop for processing, and the quality is significantly worse. Specifically, when I generate a .wav file from the python script receiving the Serial data on my laptop, I'm seeing varying amplitudes in what should be a perfect sine waves. See picture below. I'm playing a sine wave twice, and each channel has these weird amplitude differences. The interesting thing is that these differences always happen in blocks of chunk_size, at the same time on each channel. It often skips samples at the chunk boundaries too. Here's what I've tried so far:

Rather than send the data via serial, record it straight to .wav files on an SD card on Teensy. These files sound great and have no such issues.
Swap to a different .wav file writing library in my python script on my laptop. Still bad, on change.
Rather than write to a .wav file in real-time in my python script, collect all the received data in lists and write it all at the end. These files also have the exact same issue.
Checked memory usage with AudioMemoryUsageMax(), and it's all below 300.
Tried baud rates of 9600 and 115200, both are equally bad.

So, it's not the mic quality and not the .wav libraries being slow or broken. I'm thinking it should thus have something to do with how I send the data via Serial? Any ideas? Arduino code and python code are both below.

C-like:

#include <Bounce.h>
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

const int NUM_CHANNELS = 8;

AudioInputI2SOct i2s_oct1;
AudioInputI2S2 i2s2_1;

AudioFilterStateVariable filters[NUM_CHANNELS];
AudioAmplifier amps[NUM_CHANNELS];
AudioRecordQueue queues[NUM_CHANNELS];

AudioConnection patchCord1(i2s_oct1, 0, filters[0], 0);
AudioConnection patchCord2(i2s_oct1, 1, filters[1], 0);
AudioConnection patchCord3(i2s_oct1, 2, filters[2], 0);
AudioConnection patchCord4(i2s_oct1, 3, filters[3], 0);
AudioConnection patchCord5(i2s_oct1, 4, filters[4], 0);
AudioConnection patchCord6(i2s_oct1, 5, filters[5], 0);
AudioConnection patchCord7(i2s_oct1, 6, filters[6], 0);
AudioConnection patchCord8(i2s_oct1, 7, filters[7], 0);
// AudioConnection          patchCord9(i2s2_1, 0, filters[8], 0);
// AudioConnection          patchCord10(i2s2_1, 1, filters[9], 0);
AudioConnection patchCord11(filters[0], 2, amps[0], 0);
AudioConnection patchCord12(filters[1], 2, amps[1], 0);
AudioConnection patchCord13(filters[2], 2, amps[2], 0);
AudioConnection patchCord14(filters[3], 2, amps[3], 0);
AudioConnection patchCord15(filters[4], 2, amps[4], 0);
AudioConnection patchCord16(filters[5], 2, amps[5], 0);
AudioConnection patchCord17(filters[6], 2, amps[6], 0);
AudioConnection patchCord18(filters[7], 2, amps[7], 0);
// AudioConnection          patchCord19(filters[8], 2, amps[8], 0);
// AudioConnection          patchCord20(filters[9], 2, amps[9], 0);
AudioConnection patchCord21(amps[0], queues[0]);
AudioConnection patchCord22(amps[1], queues[1]);
AudioConnection patchCord23(amps[2], queues[2]);
AudioConnection patchCord24(amps[3], queues[3]);
AudioConnection patchCord25(amps[4], queues[4]);
AudioConnection patchCord26(amps[5], queues[5]);
AudioConnection patchCord27(amps[6], queues[6]);
AudioConnection patchCord28(amps[7], queues[7]);
// AudioConnection          patchCord29(amps[8], queues[8]);
// AudioConnection          patchCord30(amps[9], queues[9]);

#define SDCARD_CS_PIN BUILTIN_SDCARD

int mode = 0;  // 0=stopped, 1=recording, 2=playing

// Bounce objects to easily and reliably read the buttons
// Bounce buttonRecord = Bounce(0, 8);
// Bounce buttonStop = Bounce(1, 8);  // 8 = 8 ms debounce time

// Bounce object for the button
Bounce buttonToggle = Bounce(0, 8);  // 8 = 8 ms debounce time

// File files[NUM_CHANNELS];

const int audioMemory = 896;
const int chunkFactor = 15;

void setup() {
  Serial.begin(9600);
  SD.begin(SDCARD_CS_PIN);

  AudioMemory(audioMemory);
  if (NUM_CHANNELS <= 8) {
    for (int i = 0; i < NUM_CHANNELS; i++) {
      filters[i].frequency(60);  // filter out DC & extremely low frequencies
      amps[i].gain(6.5);         // make it louder
    }
  } else {
    for (int i = 0; i < 8; i++) {
      filters[i].frequency(60);  // filter out DC & extremely low frequencies
      amps[i].gain(8.5);         // make it louder
    }
    for (int i = 8; i < NUM_CHANNELS; i++) {
      filters[i].frequency(30);  // filter out DC & extremely low frequencies
      amps[i].gain(0.25);        // make it less loud
    }
  }

  pinMode(0, INPUT_PULLUP);
  pinMode(LED_BUILTIN, OUTPUT);  // Initialize the built-in LED pin as an output
}

void loop() {
  buttonToggle.update();

  if (buttonToggle.fallingEdge()) {
    if (mode == 0) {
      startRecording();
    } else if (mode == 1) {
      stopRecording();
    }
  }

  if (mode == 1) {
    continueRecording();
  }
}

void startRecording() {
  // for (int i = 0; i < NUM_CHANNELS; i++) {
  //   String fileString = "mic";
  //   if (i<10) { fileString += "0"; }
  //   fileString += String(i) + ".RAW";

  //   char* filePath = fileString.c_str();
    
  //   if (SD.exists(filePath)) {
  //     SD.remove(filePath);
  //   }

  //   files[i] = SD.open(filePath, FILE_WRITE);
  //   if (!files[i]) {
  //     Serial.println("❌ File creation failed: " + String(i));
  //   }
  // }

  for (int i = 0; i < NUM_CHANNELS; i++) {
    queues[i].begin();
    // printf("🎙️ NOW RECORDING: " + String(i));
  }

  digitalWrite(LED_BUILTIN, HIGH);  // Turn on the built-in LED to indicate recording
  mode = 1;
}

void continueRecording() {
  for (int i = 0; i < NUM_CHANNELS; i++) {
    if (queues[i].available() >= chunkFactor) {
      for (int j = 0; j < chunkFactor; j++) {
        Serial.write((byte*)queues[i].readBuffer(), 128 * 2); // Write a full packet. 128 samples of 2 bytes each
        // files[i].write((byte*)queues[i].readBuffer(), 128*2); // 128 samples per packet, 2 bytes each
        queues[i].freeBuffer();
      }
    }
  }
}

void stopRecording() {
  for (int i = 0; i < NUM_CHANNELS; i++) {
    queues[i].end();
  }
  // printf("✅ ALL QUEUES STOPPED");
  // Serial.println("✅ ALL QUEUES STOPPED");

  digitalWrite(LED_BUILTIN, LOW);  // Turn off the built-in LED to indicate stopping

  if (mode == 1) {
    for (int i = 0; i < NUM_CHANNELS; i++) {
      // Serial.print("writing queue: " + String(i));
      while (queues[i].available() > 0) {
        // Serial.println(".");
        Serial.write((byte*)queues[i].readBuffer(), 128 * 2);
        // files[i].write((byte*)queues[i].readBuffer(), 128*2); // 128 samples per packet, 2 bytes each
        queues[i].freeBuffer();
      }
      // printf();
    }

    // for (int i = 0; i < NUM_CHANNELS; i++) {
    //   files[i].close();
    // }
  }

  mode = 0;
}

Python:

import numpy as np
import serial
from scipy.io import wavfile
from pynput import keyboard

recording_enabled = True

def on_press(key):
    global recording_enabled
    if key == keyboard.Key.space:
        recording_enabled = not recording_enabled
        print(f"Recording {'enabled' if recording_enabled else 'disabled'}")

listener = keyboard.Listener(on_press=on_press)
listener.start()

# Set serial port and baud rate
serial_port = "/dev/cu.usbmodem142945201"
baud_rate = 9600  # Should match the baud rate in Arduino

# Set audio settings
sample_width = 2  # 16-bit audio, so 2 bytes
sample_rate = 44100
num_channels = 8

chunk_factor = 15
packet_size = 128 * sample_width
chunk_size = chunk_factor * packet_size

try:
    # Open the serial port
    ser = serial.Serial(serial_port, baud_rate)
    print(f"Connected to {serial_port}.")

    data = [np.array([], dtype=np.int16) for _ in range(num_channels)]

    while recording_enabled: # toggled via spacebar

        for i in range(num_channels):
            audio_bytes = ser.read(chunk_size)  # chunk_size bytes = 128 samples
            data[i] = np.append(data[i], np.frombuffer(audio_bytes, dtype=np.int16))
        
    for i in range(num_channels):
        wavfile.write("mic_{}.wav".format(i), rate=44100, data=data[i])

    print("✅ Done writing data to wav files")


except KeyboardInterrupt:
    print("Serial communication stopped by the user.")

except serial.SerialException as e:
    print(f"An error occurred: {e}")
finally:
    ser.close()
    print("✅ Done reading and playing data")

h4yn0nnym0u5e · Dec 14, 2023

It’s late here in UK, so can’t look in any detail, but you have no apparent method for tracking which channel is which, or dealing with data loss. The received waveforms look as if chunks are being swapped between channels.

Can you record to built in SD, then transfer when you’ve done? Otherwise you probably need to use SLIPserial or similar to transfer packets with a bit of ID and error checking.

maxnabokow · Dec 15, 2023

Hi @h4yn0nnym0u5e, that's a great idea! I'm now sending a struct that includes id and start/end markers. Works well for receiving all data in an array in the python script and writing it to a .wav file all at the end, but as soon as I go to real-time playback, things get weird. I get an invalid end marker a couple seconds into the transmission, so things are getting out of order somehow. Mind taking a look at what I've got below? Is there a better way to send ID along?

C-like:

#include <Bounce.h>
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

const int NUM_CHANNELS = 8;
const int CHUNK_FACTOR = 15;

AudioInputI2SOct i2s_oct1;
AudioInputI2S2 i2s2_1;

AudioFilterStateVariable filters[NUM_CHANNELS];
AudioAmplifier amps[NUM_CHANNELS];
AudioRecordQueue queues[NUM_CHANNELS];

AudioConnection patchCord1(i2s_oct1, 0, filters[0], 0);
AudioConnection patchCord2(i2s_oct1, 1, filters[1], 0);
AudioConnection patchCord3(i2s_oct1, 2, filters[2], 0);
AudioConnection patchCord4(i2s_oct1, 3, filters[3], 0);
AudioConnection patchCord5(i2s_oct1, 4, filters[4], 0);
AudioConnection patchCord6(i2s_oct1, 5, filters[5], 0);
AudioConnection patchCord7(i2s_oct1, 6, filters[6], 0);
AudioConnection patchCord8(i2s_oct1, 7, filters[7], 0);

AudioConnection patchCord11(filters[0], 2, amps[0], 0);
AudioConnection patchCord12(filters[1], 2, amps[1], 0);
AudioConnection patchCord13(filters[2], 2, amps[2], 0);
AudioConnection patchCord14(filters[3], 2, amps[3], 0);
AudioConnection patchCord15(filters[4], 2, amps[4], 0);
AudioConnection patchCord16(filters[5], 2, amps[5], 0);
AudioConnection patchCord17(filters[6], 2, amps[6], 0);
AudioConnection patchCord18(filters[7], 2, amps[7], 0);

AudioConnection patchCord21(amps[0], queues[0]);
AudioConnection patchCord22(amps[1], queues[1]);
AudioConnection patchCord23(amps[2], queues[2]);
AudioConnection patchCord24(amps[3], queues[3]);
AudioConnection patchCord25(amps[4], queues[4]);
AudioConnection patchCord26(amps[5], queues[5]);
AudioConnection patchCord27(amps[6], queues[6]);
AudioConnection patchCord28(amps[7], queues[7]);

#define SDCARD_CS_PIN BUILTIN_SDCARD

int mode = 0;  // 0=stopped, 1=recording, 2=playing

Bounce buttonToggle = Bounce(0, 8);  // 8 = 8 ms debounce time

const int audioMemory = 896;
const int sampleWidth = 2; // 16 bits = 2 bytes
const int packetSize = 128 * sampleWidth; // 128 samples per packet
const int chunkSize = CHUNK_FACTOR * packetSize;

const byte START_MARKER = 0x02; // STX (Start of Text)
const byte END_MARKER = 0x03;   // ETX (End of Text)
const byte END_OF_TRANSMISSION = 0x44;

struct AudioPacket {
  byte startMarker;
  byte channelId;
  byte audioData[chunkSize];
  byte endMarker;
};

AudioPacket createAudioPacket(byte channelId, const byte* audioData) {
  AudioPacket packet;
  packet.startMarker = START_MARKER;
  packet.channelId = channelId;
  memcpy(packet.audioData, audioData, chunkSize);
  packet.endMarker = END_MARKER;
  return packet;
}

void sendAudioPacket(const AudioPacket &packet) {
  Serial.write(packet.startMarker);
  Serial.write(packet.channelId);
  Serial.write(packet.audioData, chunkSize);
  Serial.write(packet.endMarker);
}

void setup() {
  Serial.begin(9600);
  SD.begin(SDCARD_CS_PIN);

  AudioMemory(audioMemory);
  if (NUM_CHANNELS <= 8) {
    for (int i = 0; i < NUM_CHANNELS; i++) {
      filters[i].frequency(60);  // filter out DC & extremely low frequencies
      amps[i].gain(6.5);         // make it louder
    }
  } else {
    for (int i = 0; i < 8; i++) {
      filters[i].frequency(60);  // filter out DC & extremely low frequencies
      amps[i].gain(8.5);         // make it louder
    }
    for (int i = 8; i < NUM_CHANNELS; i++) {
      filters[i].frequency(30);  // filter out DC & extremely low frequencies
      amps[i].gain(0.25);        // make it less loud
    }
  }

  pinMode(0, INPUT_PULLUP);
  pinMode(LED_BUILTIN, OUTPUT);  // Initialize the built-in LED pin as an output
}

void loop() {
  buttonToggle.update();

  if (buttonToggle.fallingEdge()) {
    if (mode == 0) {
      startRecording();
    } else if (mode == 1) {
      stopRecording();
    }
  }

  if (mode == 1) {
    continueRecording();
  }
}

void startRecording() {
  for (int i = 0; i < NUM_CHANNELS; i++) {
    queues[i].begin();
  }

  digitalWrite(LED_BUILTIN, HIGH);  // Turn on the built-in LED to indicate recording
  mode = 1;
}

void continueRecording() {
  for (byte channel = 0; channel < NUM_CHANNELS; channel++) {
    if (queues[channel].available() >= CHUNK_FACTOR) {
      byte chunkBuffer[chunkSize] = {0};
      byte* bufferPtr = chunkBuffer;
      for (byte packetNum = 0; packetNum < CHUNK_FACTOR; packetNum++) {
        byte* audioData = (byte*)queues[channel].readBuffer();
        memcpy(bufferPtr, audioData, packetSize);
        bufferPtr += packetSize;
        queues[channel].freeBuffer();
      }
      AudioPacket packet = createAudioPacket(channel, chunkBuffer);
      sendAudioPacket(packet);
    }
  }
}

void stopRecording() {
  for (int i = 0; i < NUM_CHANNELS; i++) {
    queues[i].end();
  }

  digitalWrite(LED_BUILTIN, LOW);  // Turn off the built-in LED to indicate stopping

  Serial.write(END_OF_TRANSMISSION);

  // if (mode == 1) {
  //   for (int i = 0; i < NUM_CHANNELS; i++) {
  //     while (queues[i].available() > 0) {
  //       Serial.write((byte*)queues[i].readBuffer(), 128 * 2); // Write a full packet. 128 samples of 2 bytes each
  //       queues[i].freeBuffer();
  //     }
  //   }
  // }

  mode = 0;
}

Python:

import config as c
import serial
from beamforming import iterative_beamforming, mix_channels, simple_delay_sum
from beamforming import calculate_mic_delays

from pynput import keyboard
import pyaudio
import numpy as np

target_index = 1
num_iterations = 15

chunk_factor = 15 # how many packets should be processed at a time
packet_size = 128 * c.sample_width # each packet is 128 samples
chunk_size = chunk_factor * packet_size

serial_port = "/dev/cu.usbmodem142945201"
baud_rate = 9600  # Should match the baud rate in Arduino
START_MARKER = 0x02
END_MARKER = 0x03
END_OF_TRANSMISSION = 0x44;

# Open a PyAudio stream using callback (non-blocking)
p = pyaudio.PyAudio()
stream = p.open(format=p.get_format_from_width(c.sample_width),
                channels=1,
                rate=c.sample_rate,
                output=True)

beamforming_enabled = False


# space key toggles beamforming
def on_press(key):
    global beamforming_enabled
    if key == keyboard.Key.space:
        beamforming_enabled = not beamforming_enabled
        print(
            f"Beamforming {'enabled' if beamforming_enabled else 'disabled'}")

listener = keyboard.Listener(on_press=on_press)
listener.start()

try:
    # Open the serial port
    ser = serial.Serial(serial_port, baud_rate)
    print(f"Connected to {serial_port}.")

    # Calculate the expected ToA (in samples) from each source to each microphone
    # Axis 0 indicates which source we are looking at
    # Axis 1 indicates the microphone that the source reaches
    # The value at each location is the expected delay in samples
    sm_delays = calculate_mic_delays(c.mic_locs, c.source_locs, c.sample_rate)

    max_delays_for_source = [max(d) for d in sm_delays]

    # num_sources x num_mics x chunk_size//2, all zeros at first
    leftovers = []
    for s in range(c.num_sources):
        max_delay = max_delays_for_source[s]
        curr_source_leftovers = [] # list of mics for a given source
        for m in range(c.num_mics):
            curr_delay = sm_delays[s][m]
            curr_source_leftovers.append([0]*(max_delay-curr_delay)) # padding for a given mic
        leftovers.append(curr_source_leftovers)

    while True:
        data = np.zeros((c.num_mics, chunk_size//2), dtype=np.int16)

        byte = ser.read(1)[0]

        if byte == START_MARKER:
            # print("✅ Received start marker, reading data from serial port")

            channel_id = ser.read(1)[0]
            audio_data = ser.read(chunk_size)

            print(f"✅ Read {len(audio_data)} bytes from channel {channel_id}")

            end_marker = ser.read(1)[0]

            if end_marker != END_MARKER:
                print(f"❌ Received invalid end marker: {end_marker}")
                continue

            data[channel_id] = np.frombuffer(audio_data, dtype=np.int16)

        elif byte == END_OF_TRANSMISSION:
            # print("✅ Received EOT, done reading data from serial port")
            break
        else:
            print(f"❌ Received invalid byte: {byte}")
            continue

        if beamforming_enabled:
            # output, leftovers = iterative_beamforming(data, sm_delays, max_delays_for_source, leftovers=leftovers,
            output, leftovers[target_index] = simple_delay_sum(data, sm_delays[target_index], max_delays_for_source[target_index], leftovers=leftovers[target_index])
            # targeted_audio = output
            stream.write(output.tobytes())
        else:
            mixed_data = mix_channels(data)
            stream.write(mixed_data.tobytes())

except KeyboardInterrupt:
    print("Serial communication stopped by the user.")
except serial.SerialException as e:
    print(f"An error occurred: {e}")
finally:
    ser.close()
    stream.stop_stream()
    stream.close()
    p.terminate()
    print("✅ Done reading and playing data")

h4yn0nnym0u5e · Dec 15, 2023

The thing about SLIPserial or similar approaches is that the packetisation done by the library prevents your start / end / special characters appearing in the data stream, so you should never lose sync, though of course you may lose packets. There's a SLIPserial module as part of the Teensy OSC library, and I'm sure something exists for Python - it's got a library for everything. Ah, yes, I've even used it - sliplib.

The packets you're using are pretty big - I make it 3843 bytes / packet, so losing one is quite expensive. Not that you currently have a mechanism to recover by requesting re-transmission, so it's "just" the loss of about 45ms of audio on at least one channel. But, if you have lost sync, then your Python falls back to reading one byte at a time and complaining about all the invalid bytes. This will be slow (according to wisdom on other threads here), and you'll probably take ages to regain sync because the next start marker you think you see will most likely actually be a byte from the audio data.

I think I'd probably be inclined to use smaller packets, say by sending 5 audio blocks (1280 bytes plus overhead) in each packet, so 3 packets per channel each time continueRecording() triggers, or 24 packets for 8 channels. Pick a strategy for IDing each packet, and have Python acknowledge correct reception, or flag any that go missing so the Teensy can re-send. Don't queue.freeBuffer() until it's confirmed the buffer got through, or just keep all 24 packets - it's less than 32k. You'd want a special short "all 24 of this batch sent" packet, in case it's the last one that goes AWOL. Re-transmission should be rare, so low overhead. Obviously Python then needs to re-assemble audio in block order, not reception order...

Hmmm .. it occurs to me that the big value for CHUNK_FACTOR is mainly to do with making SD card writes efficient, so you may not need that for serial transmission.

You might want to look at addMemoryForWrite() - I'm not 100% sure it applies to USB serial, but it may improve transmission efficiency if you have a buffer big enough to take your maximum packet size. Oh, and you don't need to worry about the baud rate matching, it's entirely irrelevant for the USB serial port.

maxnabokow · Dec 15, 2023

Okay, I'll look into SLIPserial and sliplib, thank you! Not having the markers be part of the data stream does sound nice. Do you have an example of sending via SLIP from Teensy and receiving in Python that I could look at? The documentation feels subpar.

h4yn0nnym0u5e · Dec 16, 2023

Didn't have an example, but here's something I just cooked up:

C++:

/*
 * Demo of SLIPserial-based data transfer
 */
#include <SLIPEncodedUSBSerial.h>
#include <FastCRC.h>

FastCRC32 crc;

#define DATA_COUNT 128
struct __attribute__((packed)) payload
{
  uint32_t  crc;
  uint8_t   channel;
  uint8_t   extra;
  uint16_t  count;
  uint16_t  data[DATA_COUNT];
};

SLIPEncodedUSBSerial slipSer{Serial};
payload data;
uint8_t channel = 1;

void setup()
{
  pinMode(LED_BUILTIN,OUTPUT);

  while (!Serial)
    ;

  data.channel = 1;
  for (int i=0;i<DATA_COUNT;i++)
    data.data[i] = (i % 26 + 'A') * 257;
}

uint32_t lastTx;

void loop()
{
  // Transmission
  if (millis() - lastTx >= 250)
  {
    lastTx = millis(); 
 
    data.channel = channel; // de-corrupt the packet
    data.count++;
    data.crc = crc.crc32(&data.channel, sizeof data - offsetof(payload,channel));
    if (0 == random(10)) // randomly corrupt some packets
      data.channel = 99;
    
    slipSer.beginPacket();
    slipSer.write((uint8_t*) &data, sizeof data);
    slipSer.endPacket();
  }

  // Reception
  if (slipSer.available()) // we have a packet
  {
    uint16_t rxValue;
    digitalWrite(LED_BUILTIN,1);

    // we expect a 16-bit value
    // This is not implemented in current OSC library:
    // slipSer.readBytes((uint8_t*) &rxValue, sizeof rxValue);
    rxValue  = slipSer.read();
    rxValue |= slipSer.read()<<8;
    data.extra = rxValue;

    // dump any remaining data
    while (slipSer.available())
      slipSer.read();
      
    delay(10); // allow LED to be seen!     
  }
  else
      digitalWrite(LED_BUILTIN,0);
}

Python:

import serial # need to "pip install pyserial" for this to work
import sliplib # need to "pip install sliplib" for this to work
import struct
from binascii import crc32

portName = "COM34"  # change to suit your setup

#########################################################################
# Set up communication channel
ser = serial.Serial(portName,115200,timeout=1)  # baud rate irrelevant when USB
SLIPser = sliplib.SlipStream(ser,chunk_size=1)

BadFlags = 0
while 1:
    try:
        rxPkt = SLIPser.recv_msg()
        crc, channel, extra, count = struct.unpack_from("<LbBH", rxPkt)

        # CRC check
        myCRC = crc32(rxPkt[4:])
        OK = "OK" if myCRC == crc else "not OK"
        BadFlags *= 2
        if "OK" != OK:
            BadFlags |= 1
            
        print(f"Packet: crc={crc:08X}/{myCRC:08X} - {OK}, channel={channel}, count={count}, extra={extra:02X}, size={len(rxPkt)}")

        if 0 == count % 8:  # respond every 8th packet
            SLIPser.send_msg(struct.pack("<H", BadFlags))
            print(f"Transmit {BadFlags:02X}")
            BadFlags = 0
                            
    except KeyboardInterrupt:
        break

ser.close()
print("Finished")

This uses fairly small packets, and works slowly so you can watch what's happening, but the general principles "ought to" work faster and with bigger packets. It deliberately corrupts some packets to show the CRC checks working, and sending the occasional response back from Python with flags requesting re-transmission. The response need to be fleshed out, just send flags doesn't convey all the information needed for the Teensy to re-transmit the corrupted packets - it'll need e.g. the packet count and / or channel numbers as well. There's also no check in Python that the received data is big enough for the struct.unpack_from() call to work...

maxnabokow · Dec 16, 2023

Hi @h4yn0nnym0u5e, got SLIP implemented. Naming convention of chunk vs. packet gets a bit confusing now. I'll use "chunk" for what I've been calling chunk so far (and what SLIP calls packet), and I'll use packet for what the Teensy AudioRecordQueue calls packet (so 128 int16's).

Again, things are working for a while, but then they break: The size of the chunk received stays consistently at the expected size based on the struct for a couple of seconds, then starts jumping all over the place (mostly smaller than expected). The number of chunks it takes to start acting up is exactly proportional with my chunk size. Bigger size, less chunks until size variation begins.

I've tried a couple of things:

Checked AudioMemoryUsageMax(), that's all consistently low (until after it breaks).
Thought it might be Teensy RAM, but the freeMemory function floating around on this forum tells me there's lots available. Even tried allocating my buffer dynamically with calloc/free, no changes.
Checked Teensy's Serial.availableForWrite(). Floating around 6144 and 4096, seems all good since my chunk size is small enough.
Tried just sending buffers of zero rather than getting them from the AudioRecordQueue. Same issue.
Checked ser.in_waiting in my Python script. This one's interesting. Hovers consistently around 1000 for me, before and after breaking. For context, I'm on a maxed-out MacBook Air. But when I run everything on my research partner's similarly powerful Windows laptop, his in_waiting is always in the millions, but fluctuates much more strongly. He never gets this issue of size variation that I run into. My in_waiting of 1000 is thus smaller than my chunk size, usually. Is that something to worry about?

So, this makes me (a noob) think that it's got to do with Serial buffer overflow/underflow somehow. What could be the issue? Code below.

Using SD card or adding delays to the Serial sending isn't really an option for me, I need playback on my laptop to be as close to real-time as possible. Also not worried about re-sending lost packets either. Don't think this is due to packet loss.

C:

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SerialFlash.h>
#include <SLIPEncodedUSBSerial.h>
#include <FastCRC.h>

FastCRC32 crc;
SLIPEncodedUSBSerial slipSer{Serial};

const int NUM_CHANNELS = 8;
const int CHUNK_FACTOR = 5;

AudioInputI2SOct i2s_oct1;

AudioFilterStateVariable filters[NUM_CHANNELS];
AudioAmplifier amps[NUM_CHANNELS];
AudioRecordQueue queues[NUM_CHANNELS];

AudioConnection patchCord1(i2s_oct1, 0, filters[0], 0);
AudioConnection patchCord2(i2s_oct1, 1, filters[1], 0);
AudioConnection patchCord3(i2s_oct1, 2, filters[2], 0);
AudioConnection patchCord4(i2s_oct1, 3, filters[3], 0);
AudioConnection patchCord5(i2s_oct1, 4, filters[4], 0);
AudioConnection patchCord6(i2s_oct1, 5, filters[5], 0);
AudioConnection patchCord7(i2s_oct1, 6, filters[6], 0);
AudioConnection patchCord8(i2s_oct1, 7, filters[7], 0);

AudioConnection patchCord11(filters[0], 2, amps[0], 0);
AudioConnection patchCord12(filters[1], 2, amps[1], 0);
AudioConnection patchCord13(filters[2], 2, amps[2], 0);
AudioConnection patchCord14(filters[3], 2, amps[3], 0);
AudioConnection patchCord15(filters[4], 2, amps[4], 0);
AudioConnection patchCord16(filters[5], 2, amps[5], 0);
AudioConnection patchCord17(filters[6], 2, amps[6], 0);
AudioConnection patchCord18(filters[7], 2, amps[7], 0);

AudioConnection patchCord21(amps[0], queues[0]);
AudioConnection patchCord22(amps[1], queues[1]);
AudioConnection patchCord23(amps[2], queues[2]);
AudioConnection patchCord24(amps[3], queues[3]);
AudioConnection patchCord25(amps[4], queues[4]);
AudioConnection patchCord26(amps[5], queues[5]);
AudioConnection patchCord27(amps[6], queues[6]);
AudioConnection patchCord28(amps[7], queues[7]);

const int audioMemory = 896;

struct __attribute__((packed)) AudioPacket {
  uint32_t  crc;
  uint8_t   channel;
  int32_t   available;
  int16_t   data[CHUNK_FACTOR * 128];
};

AudioPacket packet;

void setup() {
  Serial.begin(115200);

  AudioMemory(audioMemory);

  for (int i = 0; i < NUM_CHANNELS; i++) {
    filters[i].frequency(60);  // filter out DC & extremely low frequencies
    amps[i].gain(6.5);         // make it louder
  }

  while (!Serial)
    ;

  for (uint8_t channel = 0; channel < NUM_CHANNELS; channel++) {
    queues[channel].clear();
    queues[channel].begin();
  }

  pinMode(0, INPUT_PULLUP);
}

// #ifdef __arm__
// // should use uinstd.h to define sbrk but Due causes a conflict
// extern "C" char* sbrk(int incr);
// #else  // __ARM__
// extern char *__brkval;
// #endif  // __arm__

// int freeMemory() {
//   char top;
//   #ifdef __arm__
//     return &top - reinterpret_cast<char*>(sbrk(0));
//   #elif defined(CORE_TEENSY) || (ARDUINO > 103 && ARDUINO != 151)
//     return &top - __brkval;
//   #else  // __arm__
//     return __brkval ? &top - __brkval : &top - __malloc_heap_start;
//   #endif  // __arm__
// }

uint32_t lastTx;

void loop() {
  for (uint8_t channel = 0; channel < NUM_CHANNELS; channel++) {
    int numAvailable = queues[channel].available();
    if (numAvailable >= CHUNK_FACTOR) {
      AudioPacket packet;

      int16_t buffer[CHUNK_FACTOR * 128] = {0};
      int16_t* pointer = buffer;

      for (uint8_t i = 0; i < CHUNK_FACTOR; i++) {
        int16_t* data = queues[channel].readBuffer();
        memcpy(pointer, data, 128 * sizeof(int16_t));
        pointer += 128;
        queues[channel].freeBuffer();
      }

      memcpy(packet.data, buffer, CHUNK_FACTOR * 128 * sizeof(int16_t));

      packet.channel = channel;
      packet.available = Serial.availableForWrite();
      packet.crc = crc.crc32(&packet.channel, sizeof packet - offsetof(AudioPacket,channel));

      slipSer.beginPacket();
      slipSer.write((uint8_t*)&packet, sizeof(packet));
      slipSer.endPacket();
    }
  }
}

Python:

import serial
import sliplib
import struct
from binascii import crc32

class SerialAdapter:
    def __init__(self, serial_obj):
        self.serial_obj = serial_obj

    def read(self, chunksize: int) -> bytes:
        # Read 'chunksize' bytes from the serial object
        return self.serial_obj.read(chunksize)

    def write(self, data: bytes) -> int:
        # Write data to the serial object
        return self.serial_obj.write(data)

ser = serial.Serial('/dev/cu.usbmodem142945201', 115200, timeout=1)
serial_adapter = SerialAdapter(ser)
SLIPser = sliplib.SlipStream(serial_adapter, chunk_size=1)

counter = 0

CHUNK_FACTOR = 5

while True:

    try:
        rxPkt = SLIPser.recv_msg()

        format_string = "<LbL"
        crc, channel, available_t41 = struct.unpack_from(format_string, rxPkt)

        # Will unpack audio data here later

        myCRC = crc32(rxPkt[4:])
        OK = "✅" if myCRC == crc else "❌"
     
        waiting = ser.in_waiting

        print(f"{counter} - Chunk: {OK}, waiting_py: {waiting}, available_t41 = {available_t41}, channel={channel}, size={len(rxPkt)}")
        counter += 1

    except KeyboardInterrupt:
        break
    except Exception as e:
        print(f"Error: {e}")

ser.close()
print("Finished")

h4yn0nnym0u5e · Dec 17, 2023

That’s really odd. Does your research partner’s machine also suffer the breakup after a time? Having millions of bytes in_waiting sounds as if Python can’t process the chunks fast enough. You are outputting to the screen on every chunk, or every 15ms or so, perhaps that’s an issue? Maybe just output some stats every 10 chunks?

h4yn0nnym0u5e · Dec 17, 2023

My mistake, it’s faster than that, it’s 8 screen lines every 15ms. Does sound optimistic…

maxnabokow · Dec 17, 2023

His machine doesn't get the breakup, no. Very odd. Millions of bytes in there, yes, but almost immediately. Almost feels like there's something else writing to his Serial port at the same time. And even more odd then that his doesn't break up. Might be something a restart would clear up.

Either way, what else could I try on my machine? Why would it always be exactly proportional? It's like it writes a certain number of bytes and then breaks.... I struggle to even understand how it could possibly read less than the expected number of bytes. I mean, if the Teensy code sends a buffer of a hard-coded size ever time, and uses SLIP for clear start/end of packet, why would Python possibly read a packet of a smaller size? Makes no sense to me.

Any other debug things I could run that could get us more clarity?

maxnabokow said:
My in_waiting of 1000 is thus smaller than my chunk size, usually. Is that something to worry about?

Is this problematic? Just because his is in the millions and never breaks, and mine is constantly "lean" (?) like that?

h4yn0nnym0u5e · Dec 17, 2023

I have to confess at this point I’m pretty much stumped. I don’t have a Mac to try stuff on, but it sounds as if there's something very different about the USB serial driver between MacOS and Windows. Maybe there's a way to bump the MacOS buffer size up over the ~2k it appears to be?

Assuming the driver can lose characters, it could lose the middle of a chunk but preserve the start / end markers, making it come out short. But I'd also expect to see it occasionally losing an end / start pair, and getting double-length chunks as a result.

I'd still be interested to know if reducing the rate of printing to the screen is helpful, even if only to eliminate it from our enquiries.

Number of simultaneous mics possible

Member

Well-known member

Member

Well-known member

Member

Member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Well-known member

Member

Well-known member