Distorted audio when using USB input on Teensy 3.1

Status
Not open for further replies.
FYI, I have the same problem with my iPhone. The USB audio breaks after 2-3 minutes, then recovers after 10 minutes.
When I apply the patch (uncomment MACOSX_ADAPTIVE_LIMIT) I get clicks. Every 6 seconds there's a zero-gap of 3ms in the signal, which corresponds to a 22 Hz error on the sampling rate.
Any recommendation, which parameter can be further tuned to match my hardware?
 
I figured out that it is possible to generate exactly 44.1kHz by using different multipliers and divider for creating MCLK.

For example our target MCLK is 256 * 44100 = 11289600. This needs to be derived from 96000000 and can be done so with the following multiplier and divider:

(96000000 * 147) / 625 = 11289600. Using this MCLK we can get exactly 44.1 kHz, rather than slightly over it, which was causing the issue on the Mac.

Here's my patch: https://github.com/dturner/Audio/commit/77ee3c3c0548ede9cb92472532b7dbf4cbed7b1c

Initial tests seem promising, although I did get some distortion after about 10 mins, but that may be because I was building against an old version of the teensy3 library which doesn't contain Paul's bugfix to the feedback accumulator. Interested to hear your results.
 
Unfortunately after further testing it looks like this isn't fixed, it just takes longer (6-14 mins) for the audio to start distorting. Don't know how I didn't get any glitches for so long - must've been an anomaly.
 
I'm going to assume that with the new MCLK value the Teensy is consuming audio data at a rate of 44.1kHz +/- an acceptable range for the Mac to deal with. If this assumption is correct it means that the sync endpoint is not working correctly (I am not defining MACOSX_ADAPTIVE_LIMIT).

The sync feedback format is unsigned fixed binary point 10.14 (more info here under "5.10.4.2 Feedback" http://www.scaramanga.co.uk/stuff/qemu-usb/usb11.pdf). Does anyone know an easy way to convert between an int32 and 10.14 format in C++? The integer part is easy, it's the fraction (0.14) part which is causing me a problem. Any ideas?
 
I spoke to an Apple engineer about this yesterday who was able to shed some light on this problem. On the Mac host here's the process to output audio via USB:

- The audio app (e.g. Spotify) writes chunks of data (e.g. 128 frames) into large circular buffer (of about 1 second)
- The USB driver (running on its own clock) reads 44 or 45 frames of audio data from this circular buffer every USB (micro)frame
- The isochronous synch endpoint on the Teensy determines whether 44 or 45 audio frames are read every USB frame. This controls the rate at which the USB driver is moving the circular buffer's read pointer forward.
- The USB driver should also control the rate at which callbacks are made to the audio app so that the write pointer is moved forward at the same rate.

BUT for some reason the read pointer is overtaking the write pointer (probably because the USB driver is speeding up in line with the feedback value and the audio engine is unable to keep up - this is probably the ADAPTIVE_LIMIT which Paul found). This results in the USB driver reading invalid audio frames from the buffer. This is what the distortion is. It also explains why over a long period of time the distortion goes away as the read pointer goes so far past the write pointer that its reading valid (albeit very old) audio data.

How to fix? Not entirely sure. I'm pretty convinced that the sync feedback endpoint value is incorrect in some way (whether 10.14 value or bRefresh value), however, to debug this we really need to see what the Mac is seeing. Maybe writing a test audio app on OSX is the way forward.
 
Any chance you could allow me to contact this engineer? I'd love to send a Teensy with audio shield, so they can just plug it in and hear the issue.

Edit: Actually, I'd probably send two... one normal, and another with a fixed sync feedback value. Maybe I'll add a could LEDs to blink when Teensy detects a buffer overflow or underflow. When I was testing (months ago) it was this test with fixed feedback the led me to conclude the distortion must be on the Mac side. Their driver does correctly respond to the feedback and adapt +17 Hz. But minutes later, the data is filled with distortion, even though it keeps coming at the correctly +17 Hz adapted rate.

Edit again: I'm currently traveling, for Hackaday's SuperCon.... writing this from a hotel room. If you get a positive response on sending them hardware and I miss the reply, please ping me on Tuesday when I return to Portland. I would *really* like to somehow move forward on this issue!
 
Last edited:
I can report that I'm using the exact 44.1kHz software and it works much better than before, although I did see occasional glitches after long time but much less than before. This is using an iPhone 6 as source.
What I also did was removing the cap on the feedback number, and reducing the proportional control by 10x to reduce the jitter on the feedback number.
 
I'm planning to take another in-depth look at this. Currently waiting on a machine with Sierra, and focusing on several small issues to clean up before the 1.32 release.
 
Thanks for looking into this problem. For testing, I tried both @donturner's clock patch (https://github.com/dturner/Audio/com...b7dbf4cbed7b1c) and the MACOSX_ADAPTIVE_LIMIT. The clock patch worked great for me. I no longer get the slight pitch distortion nor the clicking. The MACOSX_ADAPTIVE_LIMIT seemed to have no effect on my limited testing.

One side effect of the patch, though, is that when I have both USB input and output going on at the same time, I get a regular glitch in the USB output going from the Teensy to the Computer. It occurs evenly at roughly 0.09 sec intervals. If I remove the clock patch, then this glitch (from teensy to computer) disappears. However, the original glitch (from computer to teensy) returns.

Below is the screenshot from Audacity of one such glitch.

Screen Shot 2017-01-09 at 6.23.26 PM.png
 
I think part of the problem is that because "feedback_accumulator" is cumulative, it gets so out of whack that it never quite recovers. Perhaps resetting it at some point might help. I'm not sure where or how.

Still, after reading the USB Spec section 5.10.4.2 covering feedback, if I am interpreting the jargon correctly, I think a modified approach might be better.

Allow me to give my understanding of the process, and please correct me if I am wrong, followed by code suggestions. It's not prefect, but it does get rid of the stuttering.

USB says feedback is:

desired data rate (Ff) is, relative to the USB SOF frequency

So the number is based on the rate that SOF packets are received (approx 1 SOF every 1 ms) and not how far behind or ahead we are in terms of audio data received, although the two are related. But once you fall behind, it doesn't much matter how far behind because you can't make up lost packets.

Anyway, the formula is:

feedback = cpu_ticks_per_sof * usb_samples_per_sec / cpu_ticks_per_sec

Roughly, this number is 44.1 (sample rate of 44100 Hz). Because this is not a round number, we expect the computer to send back about 9 packets of 44 samples followed by 1 packet of 45, repeating this pattern.

That's my understanding. But that's hard to translate this into code.

Because of this, as a method for calculating feedback, I suggest we keep track of the rate at which the data is actually received and compare it to our desired rate of consumption based of F_CPU and MCLK_*. If too slow, add a little to feedback; if too fast, subtract a little bit. I keep track of the data rate by counting bytes and packets between requests for feedback.

Now, the code suggestions:

The first thing is to get the rate as close to 44100 Hz as possible by implementing @donturner's patch: https://github.com/dturner/Audio/commit/77ee3c3c0548ede9cb92472532b7dbf4cbed7b1c, the relevant section for Teensy 3.2 is a change to output_i2s.cpp:

Code:
// MCLK needs to be 48e6 / 1088 * 256 = 11.29411765 MHz -> 44.117647 kHz sample rate
//
#if F_CPU == 96000000 || F_CPU == 48000000 || F_CPU == 24000000
  // PLL is at 96 MHz in these modes
  // patch from https://github.com/dturner/Audio/commit/77ee3c3c0548ede9cb92472532b7dbf4cbed7b1c
  #define MCLK_MULT 147
  #define MCLK_DIV  1250
  //#define MCLK_MULT 2
  //#define MCLK_DIV  17

Next, add some variables to keep track of data rates in usb_audio.cpp:

Code:
  uint32_t usb_audio_sync_feedback DMABUFATTR;
+ uint32_t usb_aduio_sync_count;
+ uint32_t usb_aduio_sync_count_pkt;
+ 
  uint8_t usb_audio_receive_setting=0;

In usb_audio_receive_callback(), add code to keep counts; these counts are reset by the feedback calculation code:

Code:
  	len >>= 2; // 1 sample = 4 bytes: 2 left, 2 right
  	data = (const uint32_t *)usb_audio_receive_buffer;
  
+ 	usb_aduio_sync_count += len;
+ 	usb_aduio_sync_count_pkt++;
+ 
  	count = AudioInputUSB::incoming_count;
  	left = AudioInputUSB::incoming_left;
  	right = AudioInputUSB::incoming_right;

Remove setting usb_audio_sync_feedback:

Code:
  	if (f) {
  		int diff = AUDIO_BLOCK_SAMPLES/2 - (int)c;
  		feedback_accumulator += diff / 3;
! 		// uint32_t feedback = (feedback_accumulator >> 8) + diff * 100;
  #ifdef MACOSX_ADAPTIVE_LIMIT
  		if (feedback > 722698) feedback = 722698;
  #endif
! 		// usb_audio_sync_feedback = feedback;
  		//if (diff > 0) {
  			//serial_print(".");
  		//} else if (diff < 0) {

Next, add to usb_dev.c, the variables we added in usb_audio.cpp:

Code:
extern uint32_t usb_aduio_sync_count;
extern uint32_t usb_aduio_sync_count_pkt;

And the heart of the calculation is performed when feedback is requested in usb_dev.c

Code:
  				b->addr = usb_audio_receive_buffer;
  				b->desc = (AUDIO_RX_SIZE << 16) | BDT_OWN;
  			} else if ((endpoint == AUDIO_SYNC_ENDPOINT-1) && (stat & 0x08)) {
+ 				// Send back how many samples the computer should send between every SOF packet (which is approx 1ms).
+ 				// Returned in 10.14 format (hence all the ">> 14" below).
+ 				// Formula is: cpu_ticks_per_sof * usb_samples_per_sec / cpu_ticks_per_sec
+ 				// In a perfect world, the answer is 44.1 packets per SOF cycle (or ~722534 in 10.14 format).
+ 				// But this is not the case because clocks never run precisely in sync.
+ 				const uint32_t usb_audio_sync_clock = 44100; // Actually: F_CPU * MCLK_MULT / MCLK_DIV / 256;
+ 				const uint32_t usb_audio_sync_clock_min = (usb_audio_sync_clock << 14) / 1000; // min samples per packet
+ 				const uint32_t usb_audio_sync_clock_max = ((usb_audio_sync_clock+5) << 14) / 1000; // max samples per packet
+ 
+ 				// Get the average number of samples per packet, should be approx 44.1 (in 10.14 format)
+ 				uint32_t avg_samples_per_packet = (usb_aduio_sync_count << 14) / usb_aduio_sync_count_pkt;
+ 				// Is it in the range of what we want?
+ 				if (avg_samples_per_packet < usb_audio_sync_clock_min) { // too low, adjust up
+ 					usb_audio_sync_feedback += 10;
+ 					if (usb_audio_sync_feedback > 722698) usb_audio_sync_feedback = 722698;
+ 				}
+ 				else if (avg_samples_per_packet > usb_audio_sync_clock_max) { // too high, adjust down
+ 					usb_audio_sync_feedback -= 10;
+ 					if (usb_audio_sync_feedback < 722370) usb_audio_sync_feedback = 722370;
+ 				}
+ 				usb_aduio_sync_count = usb_aduio_sync_count_pkt = 0;  // reset the counts
+ 
  				b = (bdt_t *)((uint32_t)b ^ 8);
  				b->addr = &usb_audio_sync_feedback;
  				b->desc = (3 << 16) | BDT_OWN;

I hard code the value of "usb_audio_sync_clock" because I don't have access to MCLK_MULT & MCLK_DIV from here.
 
I think I've just encountered this.
Audio Lib to USB to Mac OS Sierra
Current standard Arduino and Teensyduino.
Clicks every 35 ms... occasionally 70 ms.

Does this sound like the same issue?

It's not in my way, really, but I'd be happy to participate in test fixes.
Audacity - click every 35 ms.jpg
Top line is trumpet mouthpiece buzz.
Bottom line is sine wave with same frequency.

Dave
 
I wonder if part of the problem is that the input and output streams are not synchronized. From Apple, all streams for the same device must follow the same rate exactly: https://developer.apple.com/library/content/technotes/tn2274/_index.html

Unified Engine Model

AppleUSBAudio represents the streams of a USB audio device to the application layer using IOAudioFamily's IOAudioStream entities. One or more streams are associated with each engine.

In Mac OS v10.5.6 and earlier, AppleUSBAudio restricted each engine to one stream only. Then in v10.5.7, AppleUSBAudio’s engine model was redesigned to combine multiple streams per engine when possible. In cases where input and output streams can reside on the same engine, this architecture achieves sample synchronization and consistent latencies between streams.

In order to combine multiple streams on one engine, the following criteria must be met by the stream interfaces:

All sample rate sets must match.
The same synchronization type is used for all interface alternate settings.
The streams reside in the same clock domain.
Please note the following behaviors of streams that occupy the same engine:

These streams operate in the same time domain and sample rate setting (but not necessarily the same format.)
These streams cannot be activated independently. They will always start and stop together.

usb_audio_transmit_callback() always send 9 packets of 44 samples per 1 of 45, whereas usb_audio_receive_callback() receives at a slightly different rate based on the feedback. I made a slight modification so usb_audio_transmit_callback() always sends back the same number of samples it receives from the last packet in usb_audio_receive_callback(). That seemed to eliminate the clicking that starts after 10 minutes or so.
 
If anyone is still watching this thread, please give the latest beta a try and let me know how it works with your mac?

https://forum.pjrc.com/threads/42766-Teensyduino-1-36-Beta-3

This version makes a change to the input stream setting, suggested by an engineer at Apple, which is supposed to force macos to process the 2 streams separately. It seems to work well with my Macbook Air running Sierra.
 
I'd like to try it.
Is there a clean back-out process to get back to my current Arduino (1.8.1) & Teensyduino (1.35)?
 
I've tried it for over 1 hour on my system with `uname -a` below and it works flawlessly. I will run more tests later today.
Code:
Darwin 16.4.0 *** Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu-3789.41.3~3/RELEASE_X86_64 x86_64

I was using code below which takes USB input and directs it to the audio board and to USB output. I also checked the input coming from the Teensy to the computer with Audacity and it seemed to come in without any artifacts.

Code:
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// GUItool: begin automatically generated code
AudioInputUSB            usb1;           //xy=201,123
AudioOutputUSB           usb2;           //xy=201,123
AudioOutputI2S           i2s1;           //xy=417,121
AudioConnection          patchCord1(usb1, 0, i2s1, 0);
AudioConnection          patchCord2(usb1, 1, i2s1, 1);
AudioConnection          patchCord3(usb1, 0, usb2, 0);
AudioConnection          patchCord4(usb1, 1, usb2, 1);
// GUItool: end automatically generated code

AudioControlSGTL5000     sgtl5000_1;     //xy=223,285

void setup() {
  AudioMemory(10);
  sgtl5000_1.enable();
  sgtl5000_1.volume(0.5);     // set the main volume...
}

void loop() {
}
 
Last edited:
Is there a clean back-out process to get back to my current Arduino (1.8.1) & Teensyduino (1.35)?

No, but you can just unzip a clean copy of 1.8.1 and then install 1.35 onto it.

Or you can extract a clean copy of 1.8.1, rename it, run it at least one to pass the "downloaded from the internet" warning, and then run this beta installer. Arduino is designed so you can have multiple copies on your computer. They will share the same settings (and Arduino's boards manager settings) and your sketchbook folder, but each will be separate as far as Teensyduino is concerned. Best to run only one copy at a time...
 
Any news on this problem? I have the same issue in Windows since we modified the code to fit 4 channels audio (microphone) and 2 channels audio (speaker).

I wonder where I should look for the correct fix.

Thanks!
 
Any news on this problem?

Fixed long ago, around March 2017.

Raspberry Pi with Jack is known to have issues, but Macintosh works fine now, and has for at least the last year. This is an old thread about the problem that's now just old history.

FWIW, this problem *never* happened with Windows or normal desktop Linux (eg, running pulseaudio). If you're having trouble on Windows, that's something unrelated.
 
Documentation needs updating in Audio System Design Tool

Fixed long ago, around March 2017.

Raspberry Pi with Jack is known to have issues, but Macintosh works fine now, and has for at least the last year. This is an old thread about the problem that's now just old history.

FWIW, this problem *never* happened with Windows or normal desktop Linux (eg, running pulseaudio). If you're having trouble on Windows, that's something unrelated.

The documentation for the USB output object still says there is a Mac issue
 
Status
Not open for further replies.
Back
Top