High quality audio streaming using NRF24 (working) - Sample rate synchronization

Dear all,

Backstory below.

What already works
High quality (16 bit @ 44.1 kHz), low latency (~6 ms) audio stream from one Teensy to another using NRF24L01 modules.

What's my issue
Naturally, the sample rates of both Teensys are not synchronized and also not exactly the same. For the audio stream this means that they are slowly drifting apart and every 3.5 s or so, a few buffers are lost (I reduced the buffer size to 64 samples to achieve a lower latency, using the 128 sample buffer, the short drop-outs happen every 7 s - latency increases to ~12 ms).

The short drop-outs are clearly audible while transmitting a sine signal, however for music it's not even a big problem (music is basically just noise - hehe).

Anyway, if possible I'd like to fix or at least reduce it and here I need your help.

I think there are 2 ways of fixing this issue:
1. Synchronizing the sample rate of the receiver to the incoming data packets
2. Accurately measuring both sample rates and resampling the data using Sample Rate Conversion (This is typically used in DSPs to align digital audio data)

Looking into the AudioStream library, I don't see a way to alter the sample rate, or is there?
Also sample rate conversion is a quite involved operation. What do you think?

Hardware Transmitter (picture attached)
Teensy 4.0 with AudioShield and NRF24L01 module

Hardware Receiver
Teensy 3.5 with AudioShield and NRF24L01 module
(Just for testing I added a LiPo battery that I could walk around with it and a mute button)

Software Transmitter
I implemented an audio object (AudioNrf24Tx) which is connected to the I2S input of the audio shield (external 3.5 mm TRS socket). In the update() method, I just copy the buffer into a global vector. The buffer is then transmitted in 16 sample packets (NRF24 has a 32 byte payload = 16 sample á 16 bit, 2 bytes) with a delay of 300 us. The delay is necessary for the NRF24 to operate stable, I saw some weird carrier frequency drifting when the delay was shorter. Fortunately, 300 us is just enough to transmit all 4 packets before the next update() (buffer has 64 samples @ 44.1 kHz, update is triggered every 1.45 ms). The audio source can be switched between a sine synth block or the line input using a button. There is also a simple frequency hopping algorithm implemented which would allow higher RF power (at least in my region, EU), but it is currently disabled for easier testing (works however).

Software Receiver
The NRF24 module pulls the interrupt pin LOW whenever data is ready to read by the Teensy. Since 1 buffer is split into 4 transmission packets, I'm using a ping-pong buffer too handle the incoming data. If one buffer is filled with valid samples, it is released to the custom audio object (AudioNrf24Rx) to be output while the incoming packets are filling the second buffer in the meantime. In the update() method not much happens, only the data is copied into audio_block_t and the buffer is set to all 0.0. This helps a lot during debugging on the oscilloscope, however might not be the best way to mask missing buffers.

Backstory
Being frustrated with the "cheaper" (200-300€) UHF in-ear monitoring systems, I started to experiment a bit with the stuff I had on hand. Honestly, I was surprised that these data rates were even possible with the NRF24 modules. For in-ear monitoring on stage, a low latency is probably most important, since you are hearing your voice in your head and also through the headphones. In my experience <10 ms is fine but above that you get phasing issues and it sounds very unnatural. My cheap system not only has a lot of drop-outs and distortions, but also the audio quality is very poor. The current stage of this project is already much better and should work for band practice, where I need about 3 m/10 ft of range. Since I have now more time for such projects (thanks to covid19), I thought about using a second NRF24 module at the receiver to build a poor-persons diversity receiver. With a certain distance between the antennas, it could help to improve transmission stability. Any thoughts on that?

I'm looking forward reading your ideas and suggestions!

Best wishes,
Ernst.
 

Attachments

  • 1_HarwareSetup.jpg
    1_HarwareSetup.jpg
    226.1 KB · Views: 324
  • 2020-04-04_PjrcForum.zip
    829 KB · Views: 240
Perhaps you could adjust the rate of the RX T3.5's I2S MCLK (with I2S0_MDR register) based on the fill level of a receive FIFO? It would form a kind of a PLL with the fill level as the phase detector. Then the RX sample rate would match the TX sample rate --- at least on average.

I'm not sure that I2S0_MDR provides a fine enough granularity for this to work, but it might be worth looking into.
 
The root cause is that T4.x clocks generate exact 44.1kHz and T3.x only approximate it as gfvalvo has noted. Even if his suggestion works, you will only space the dropped / extra packets out further.

I had the same issue in my Ethernet library https://forum.pjrc.com/threads/58660-Ethernet-audio-library-ready-to-beta-test - and now am using only T3s OR T4s on the network at any one time.

The result is almost no dropped packets at all (< 1:10000), and this seems mostly to be due to issues other than timing - network congestion, and other ethernet mishaps, mostly.
 
Thanks a lot for your suggestions! I will try using the same Teensy versions and post my results then.

@gfvalvo: I'm afraid the resolution is not high enough, the difference between the sample rates is about 0.9996.
 
Even if his suggestion works, you will only space the dropped / extra packets out further.
Actually, it will work perfectly if properly implemented. It's done all the time in the digital telecom world. The "PLL" must synch the RX frequency exactly to the TX frequency --- but only on average. Temporary variations are absorbed by the FIFO. So, as long as the PLL's bandwidth is low enough to provide acceptable jitter performance and the FIFO is large enough, you'll get glitch-free, error-free data transfer.

Thanks a lot for your suggestions! I will try using the same Teensy versions and post my results then.
@gfvalvo: I'm afraid the resolution is not high enough, the difference between the sample rates is about 0.9996.
Thought that might be the case. It could be done using an external, numerically-controlled oscillator to provide an input MCLK to the T3.5. The control for this oscillator would be derived from the FIFO fill level and an appropriate digital low pass filter.
 
Dear Ernst,

first of all, thank you for sharing your code. We are using a modified version of your code for a university project. For the project, I am developing an audio shield for the Teensy 3.2 with connectors for the NRF24l01+. The version we are using works so far but there are still dropouts that happen sometimes when the orientation of the antenna changes. We tried many different antenna boards with onboard and external antennas, using different channels, extra power supply, decoupling caps, sending at different power levels etc., but nothing helped so far to get rid of the problem. Although our problem is not related to the sample rate synchronisation I would be happy to hear if you could manage to establish a stable audio connection, and which hardware you are using.

Best,

Pascal
 
Hi Pascal,

Im glad to hear that my implementation has some use! In the meantime I managed to get an OK transmission. I will provide more details in the evening, but in the meantime you can check out my latest code.

I could not manage to transmit more data, as CD quality MONO is pretty much the limit of the NRF24 (as I use them..). Ideal would be some kind of error detection, re-transmission or even compression.

Hardware details follow.

All the best!
Ernst.

View attachment Nrf24Audio.zip
 
So, now I have a bit more time.

To start with: even with two Teensy 4.0 + Audio shield, I had some audible distortions now and then, but it was way less then my initially described problem, so I lived with it. In the end I implemented a diversity receiver (simply 2 NRF modules and some software to decide which packet to use) and the results were OK. If you have a line of sight or even outdoors, 10m should work pretty stable.

I designed a simple PCB and ordered it from JLCPCB, which just holds all the modules (see attachment). Hardware is identical for TX and RX side, however on TX side only one NRF module is used.

IMG_Bodypack_open.jpeg

Most interesting for you is probably the NRF24:
https://de.aliexpress.com/item/32687064881.html?spm=a2g0s.9042311.0.0.1d1b4c4dXQ0W7p

I used them to be able to connect IPX antennas, like these for example:
https://lcsc.com/product-detail/Antennas_DreamLNK-F5-2-4G_C403726.html

And also the very basic antennas which often come with the power-NRF24 modules.

I did a lot of testing and many measurements with a VNA and my conclusion was that PCB antennas (and the ones above) are really well tuned and work well, the black plastic antennas are often just a wire inside and pretty useless (stay away from the very short ones, they are not tuned at all).

In conclusion I can recommend such Wifi antennas (which internally have a PCB antenna):
https://www.amazon.com/Antenna-RP-SMA-2-4Ghz-Extension-Notebook/dp/B07QKF18KM

Some additional features you will find in the SW:
> Auto selection of NRF module A or B, which ever has a stable connection
> RGB LED to indicate RF receiption
> Mute button
> Auto-mute in case too many packets get lost
> TX: Switch to select between MONO (L + R) or balanced (L - R) input
> RX: Volume knob (purely SW)
(> Channel hopping is there, worked at some point but I'm not sure if it still does..)

I hope this is useful to you in some way! :)

Greetings!
Ernst

View attachment IEM3000Bodypack_pcb.pdf
View attachment IEM3000Bodypack_sch.pdf
 
As Paul discovered, the T4 audio output rate is tunable. I use this with a "one per second" character from the PC t to match the teensy clock to the highly accurate (ntp/chrony) PC clock. But it could match to any other clock, including a T3.
 
This is an awesome project. Thank you for sharing.
Do you happen to have a simpler version of the code that I could use in my Walkie Talkie project? Your current version is quite complicated for me to grasp :)

Thanks
 
Dear Ernst,

thank you so much for your long answer and for sharing your project. Somehow I only got a notification for the latest reply.

I hope this is useful to you in some way! :)

Your information is indeed very useful for me. Especially the antenna module with the small antenna connector seems very suitable for our needs (it can even be SMD assembled by JLCPCB).

We actually used the small black antennas so far but I could not notice much difference to the little longer ones which came with the antenna boards. Unfortunately using two antennas on the receiver side is not really an option for the project as we need to keep the PCB as small as possible. I think I will spend some time trying to deal with the dropouts on the software side. As we don't need the full audio frequency range I thought about downsampling the signal before transmission and upsampling it on the receiver side, unfortunately, I could not get this working yet. Or some sort of interpolation for missed packets. Have you tried this yet? There are some hints in your code that you wanted to implement this at a later stage.

Currently, I am developing a new version of the PCB (audio board with antenna). Once it is finished and I did some testing with it I am happy to share the design here.

Another question: so far we are using the Teensy 3.2 For both Tx and Rx. Could you notice any difference when using the Teensy 4?

Best,

Pascal
 
This is an awesome project. Thank you for sharing.
Do you happen to have a simpler version of the code that I could use in my Walkie Talkie project? Your current version is quite complicated for me to grasp :)

Thanks

What does not work for you? I can try to help If you are struggling to understand specific parts or cannot get the code to work.

One important thing: you need to change the block size (AUDIO_BLOCK_SAMPLES) of the teensy audio library to 64 for the code to work properly.

Best,

Pascal
 
What does not work for you? I can try to help If you are struggling to understand specific parts or cannot get the code to work.

One important thing: you need to change the block size (AUDIO_BLOCK_SAMPLES) of the teensy audio library to 64 for the code to work properly.

Best,

Pascal

I tried compiling your code as I downloaded it and I get the following error:

Code:
In file included from C:\Users\...\Downloads\Nrf24Audio (1)\Nrf24Audio\Nrf24AudioTx\Nrf24AudioTx.ino:12:0:
C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\FastLED/FastLED.h:14:21: note: #pragma message: FastLED version 3.003.003
 #    pragma message "FastLED version 3.003.003"
                     ^
In file included from C:\Users\...\Documents\Arduino\libraries\RF24/RF24_config.h:66:0,
                 from C:\Users\...\Documents\Arduino\libraries\RF24/RF24.h:18,
                 from C:\Users\...\Downloads\Nrf24Audio (1)\Nrf24Audio\Nrf24AudioTx\Nrf24AudioTx.ino:11:
C:\Program Files (x86)\Arduino\hardware\teensy\avr\cores\teensy4/FS.h: In member function 'virtual void File::whoami()':
C:\Users\...\Documents\Arduino\libraries\RF24/utility/Teensy/RF24_arch_config.h:22:16: error: 'class usb_serial_class' has no member named 'Serial'
 #define printf Serial.printf
                ^
C:\Program Files (x86)\Arduino\hardware\teensy\avr\cores\teensy4/FS.h:83:10: note: in expansion of macro 'printf'
   Serial.printf("  File    this=%x, f=%x\n", (int)this, (int)f);
          ^
C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\SD\src/SD.h: In member function 'virtual void SDFile::whoami()':
C:\Users\...\Documents\Arduino\libraries\RF24/utility/Teensy/RF24_arch_config.h:22:16: error: 'class usb_serial_class' has no member named 'Serial'
 #define printf Serial.printf
                ^
C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\SD\src/SD.h:70:10: note: in expansion of macro 'printf'
   Serial.printf("   SDFile this=%x, refcount=%u\n",
          ^
Error compiling for board Teensy 4.0.

Do I need to change anything in the code to make it compile?
I am using the latest Teensydoino and Arduino IDE.

Thank you
 
Do I need to change anything in the code to make it compile?
I am using the latest Teensydoino and Arduino IDE.

Thank you

It is the code from Ernst. I am using the first version he posted (with one receiver) but the other version you tried also compiles on my machine. Seems like some library mismatch between the teensy libraries and the RF24 library you are using. Which versions are you using?
 
It is the code from Ernst. I am using the first version he posted (with one receiver) but the other version you tried also compiles on my machine. Seems like some library mismatch between the teensy libraries and the RF24 library you are using. Which versions are you using?

I tried both codes (the older one in post #1 and the one in post #7) but both give me the same error. I use the latest NRF24 version from TMRH.
 
If I comment line 22 of C:\Users\Paul\Documents\Arduino\Libraries\RF24\utility\Teensy\RF24_arch_config.h, like this...
Code:
//#define printf Serial.printf
...the sketch Nrf24AudioTx.ino compiles without error. Didn't verify though whether the sketch is actually working.
Using library RF24 version 1.4.1 from TMRh20.

Paul
 
Back
Top