Audio board recording while playing - sampling frequency problem

Status
Not open for further replies.

CraigF

Well-known member
I have a project that uses Teensy 3.6 to acquire a half-dozen channels of sensor data and transfer the data in real time via USB Serial to a PC, where I wrote a multi-channel scrolling console to display it. One of my sensor channels is audio, and for that I use the teensy audio board. Since my serial bandwidth is limited to 3-4 Mbps and I'm primarily interested in voice-codec quality sound, I wrote a queue object that decimates the audio to 8820 Hz before applying a 256 sample FFT and transmitting everything. One of the display channels is a spectrogram of the first 45 bins of the audio signal.

All of this works perfectly (photo at bottom). Now here's my question/problem:

Periodically I also use the Teensy audio board to play a sound from an SD card. Recording continues during this playback, and until today I had thought the simultaneous recording and playback was working fine, too.

Today I connected a test microphone on a long wire that introduced more 60 Hz noise than usual. On the spectrogram (photo below) I immediately noticed that while the sound is playing, the 60 Hz / 120 Hz noise in the microphone line is shifted up by one FFT Bin, which in this case is about 34 Hz. Of course, once I noticed it in the 60Hz line, I can easily see that the same frequency-shift is affecting the actual recorded audio as well. I didn't notice it previously because I was looking at low-level periodic signals that span multiple bins in that same region.

Increasing the playback time to approximately one second it then became obvious that while the audio board is playing back sound, the rate at which audio data samples are received from the teensy drops noticeably: the scrolling displays for those channels slow down momentarily while all other channels continue to supply data at the expected rate. This is not just a delay in data transmission; there are permanently missing samples that gradually cause the audio-based channels to drift behind the other channels.

Since the 60 Hz noise source is fixed, I conclude that the frequency shift is an artifact caused by the audio board failing to keep up with its 44.1 KHz sample rate during the time when audio playback is occurring. This brings me to my question:

Before I start tearing things apart, has anybody seen this before?

Can anybody confirm that they have used the teensy audio board to accomplish one second of mono audio playback at 44.1 KHz with simultaneous recording at 44.1 KHz and NOT seen any problem with the sampling frequency on the recording? The chip documentation suggests that this should work, but there's a lot of library between me and the chip :)

The FFT is not coming from the Audio board -- I'm just doing that on the teensy itself, so the audio board does nothing but record, decimate, and periodically play a sound.

The sound file comes from an SD card on the teensy itself, but I think problems there should slow down the whole loop, affecting other data channels as well as the audio channel.

Perhaps I need to move decimation out of the audio library, where I've implemented it as a queue, and handle it downstream myself?

Here is an image of the console with the audio spectrogram running and an image of the spectrogram showing the obvious shift in the 60 Hz / 120 Hz recorded signal every time a sound is played.

Any thoughts about where to start looking, or how to work around this?

spectrogram_shift.png
ws_console.jpg
 
The audio board itself does not do decimation. The Audio library can.

Post your code - preferably in code tags using the # icon.

You could use the monitoring functions to check on the memory and cpu usage of the audio library to see if either, or both, are too heavily used. See the example in the IDE - File|Examples|Audio|MemoryandCpuUsage

Pete
 
The audio board itself does not do decimation. The Audio library can.

Post your code - preferably in code tags using the # icon.

You could use the monitoring functions to check on the memory and cpu usage of the audio library to see if either, or both, are too heavily used. See the example in the IDE - File|Examples|Audio|MemoryandCpuUsage

Pete

Thanks for that suggestion, Pete. I have turned on instrumentation on memory and CPU. When recording is enabled and playback is disabled I see AudioProcessorMax of 31 and AudioMemoryMax of 8. With both recording and playback enabled I see AudioProcessorMax of 105 and AudioMemoryMax of 10.

The documentation says that AudioProcessorUsageMax is an estimate of the maximum percentage of time any audio update has ever used. I don't know what to do with a percentage of 105, but at a guess this indicates the estimation algorithm isn't well bounded, and I should interpret this as having maxed out the processor for some time period? If so, that seems like a possible mechanism for missing audio samples.

I have a hard time thinking this amount of processing would be a problem, given the intensity of other projects people are doing with the board and library.

The maxed-out cpu must be for a very brief period, because there are another five non-audio sensor channels all acquiring data, running algorithms, and transmitting it all to the PC with nary a hiccough. Loop times are fast and stable, and I'm requesting audio buffers much more often than I receive them.

Yes, when I said "audio board" I meant "audio library interacting with the board:)
Though actually, the library itself didn't do decimation the last time I looked -- it included a function referred to as decimation, but when I looked at the code it was not actually decimation in the sense of limit filtering followed by down-sampling. Perhaps the library has changed, but in my case I perform filtering and down-sampling to 8820 Hz in a queue object that I wrote. As you suggested, the queue object does execute in the context of the audio library, and perhaps that's responsible for some of the audioprocessor usage. I'll move it out of the audio library as part of my testing if I don't find any other hints.

I didn't post code because I'm not (yet) asking what's wrong with my code. I'm still hoping to learn whether anybody else has performed playback during recording, and whether they know for sure that they did or did not experience a reduction in the availability of audio buffers below what they should have received at a sampling rate of 44.1 KHz.

I have been running this system for a year and didn't notice the problem until I happened to be looking at the spectrogram. If somebody were playing sound continuously while recording something else they'd likely never notice that their recorded audio was frequency-shifted a few Hz. You'd almost certainly need to be recording a known frequency while stopping and starting playback of audio to notice it.

My hope at this moment is for somebody who has done simultaneous playback and recording to tell me that they have or have not encountered this problem.

If I post code it will be a simplified repro, and that will come after an investigation. Right now I just hope to hear somebody else's experience to help guide my direction.
 
Last edited:
Which kind of filter do you use in the decimation and are you using DSP instructions to speed up the processing?

Does the audio output have to be at 44k1Hz? If not, you might be able to get away with reducing the sampling rate of the audio object, for example to 11025Hz or even 8000Hz. This would, I hope, remove the need for the decimation which would free up some CPU time and the audio output wouldn't be such a burden. I'm not sure if the audio playback from the SD card will work at anything other than 44k1, but if it will (or if you can "roll your own") it would be worth a try.
The sampling rate can be changed using the setI2Sfreq function in the message at:
https://forum.pjrc.com/threads/4233...ime-processing?p=175928&viewfull=1#post175928
e.g. call it like this in the setup function:
Code:
  setI2Sfreq(8000);

Pete
 
Which kind of filter do you use in the decimation and are you using DSP instructions to speed up the processing?

Does the audio output have to be at 44k1Hz? If not, you might be able to get away with reducing the sampling rate of the audio object, for example to 11025Hz or even 8000Hz. This would, I hope, remove the need for the decimation which would free up some CPU time and the audio output wouldn't be such a burden. I'm not sure if the audio playback from the SD card will work at anything other than 44k1, but if it will (or if you can "roll your own") it would be worth a try.
The sampling rate can be changed using the setI2Sfreq function in the message at:
https://forum.pjrc.com/threads/4233...ime-processing?p=175928&viewfull=1#post175928
e.g. call it like this in the setup function:
Code:
  setI2Sfreq(8000);

Pete

Thanks so much for the thoughts and suggestions. I was using arm_fir_decimate_q15 and a 72-tap filter. In response to your question I changed it to arm_fir_decimate_fast_q15, which reduced AudioMaxCPU from 105% to 94%. I was excited to see that improvement, but unfortunately the sample rate still drops by about 5% when I call for a brief period of playback. When playback is suppressed, everything is great.

I'll try a shorter filter next, but if I have playback turned off, recording with decimation only shows max audio cpu of 15% so I'm doubtful that I can gain enough by further tweaking decimation. It seems to be the combination of recording plus playback that's expensive. I'm afraid the interleaving of recording and playback may be a problem either deep in the library or (more likely) at the chip level.

I seem to have plenty of actual CPU headroom. My typical loop time is only about 600 microseconds, jumping to about 7 milliseconds for one cycle every 32 msec (when enough audio data has been received for me to perform FFT and other filtering, which is done using arm_math and outside the audio library).

PJRC.com documentation of the teensy audio board's use of interrupts is still a stub, but I'm not using any timers / isr callbacks, so that shouldn't be a problem. I used to sample my audio directly at 8KHz using the teensy ADC, but my current board revision no longer has the mic preamp and analog filters to support that and all the other channels are sampled from the loop at 75 Hz or less.

My code does already include FrankB's routine for setting the audio board sampling rate and I previously found that I could successfully record and play back samples at any desired rate, but there's no apparent way to sample at one rate and play back at another. It's no problem to generate my output files at 8K, but unfortunately this device is used for investigations of subcortical auditory brain stimulation (neuromodulation). Part of the investigation is to determine which frequencies are important, so I need to deliver the full 20 Hz to 20KHz range, which means I'm stuck with 44.1 or thereabouts.

Your suggestion made me think, though: if the chip can switch rates quickly enough (and without needing a reset) then I could modify the library recording and playback routines to set their own sampling rates. You don't happen to know whether rapid changing of sample rate is possible? I didn't see anything in the datasheet, but it's pretty long and I may have missed a note somewhere.

I was looking at alternatives and noticed that the WM8731 datasheet claims the ability to sample and play back simultaneously at different rates. I think it may need to be in master mode to do so, and I don't think the teensy audio library fully supports slave mode, but likely somebody has made it work by now, or I can give it a try. It looks like if I were to switch to the proto audio board in master mode I'd have to change the crystal to give the audio library the correct sample rate. Is that correct?

Changing the audio board would mean an entirely new device board revision for me, and probably a couple of months delay so I'd like to avoid it, but I'm beginning to fear that may be my only option.

Grasping at straws:

Do you think AudioCPU consumption would be less if I installed the serial flash memory chip and copied my files to it? I looked at the audio library code and it sort of seemed as if the playback code might be blocking on data from the SD card, but I wasn't able to tell for certain. I'll need to add another layer of file management logic, but I'll try it unless you know those reads won't block the recording routines.

It has occurred to me that perhaps I could use different decimation parameters during periods of playback. The sensible way to do that would be to know what the *actual* sampling rate was, and adjust decimation as needed to yield 8820 Hz from the true rate. Do you know of a straightforward way to get the true sampling rate from the audio library? Since I receive a full buffer of samples later than expected, I suppose I could look at elapsedMicros and calculate a sampling rate for each buffer...

Have you (or anyone else???) any other thoughts or suggestions of things I might try before going down the path of abandoning the teensy audio board?

:cool:
 
I doubt that the chip's sampling rate can be switched back and forth but I've never tried it.

If you could store the audio files in serial flash it might make a difference compared to using uSD.
Which uSD are you using? The one on the audio board or the one built into the T3.6?

Audiostream.h has this definition of the exact sampling rate:
Code:
#define AUDIO_SAMPLE_RATE_EXACT 44117.64706

Pete
 
I doubt that the chip's sampling rate can be switched back and forth but I've never tried it.

If you could store the audio files in serial flash it might make a difference compared to using uSD.
Which uSD are you using? The one on the audio board or the one built into the T3.6?

Audiostream.h has this definition of the exact sampling rate:
Code:
#define AUDIO_SAMPLE_RATE_EXACT 44117.64706

Pete

I was using the SD card on the teensy itself, but I did try switching to the one on the AudioAdaptor, and that made no difference.

Since your last suggestion, I added the serial flash memory and moved the files to it, and now AudioCPU Max is down to just 21, and AudioMemoryMax is 9 buffers.

For most loop cycles AudioProcessor usage is 1, and then when a sound is playing it jumps to 15 for a few cycles, then back down to 1.

Sadly, this did not make any difference at all to the behavior of the system: while playing at 44.1KHz it still fails to keep up with the sampling frequency of 44.1KHz.

You've helped me a lot, since I've been able to reduce audio CPU pressure and prove to myself that the cause of my problem lies elsewhere.

I'm going to try with a new (simpler) post in the support forum just to see if anybody has ever used this board and library for continuous recording with intermittent playback and verified that the recording sample rate was stable.

If nobody steps up with that information, I'll try to repro this in a stripped down project -- if I can still see the problem then I'll start digging into the library or bypass the library and test the raw board.

Thanks!
 
I doubt that a new thread is going to make any difference.
You'll be asked for code which demonstrates the problem, so work on the repro code and post it here.

Pete
 
Status
Not open for further replies.
Back
Top