DC offset removal noise/distortion in input_adc.cpp, input_adcs.cpp

I realized this is the appropriate place for the thread I had previously started over here:
https://forum.pjrc.com/threads/45695-DC-Offset-Removal-in-input_adc-cpp-input_adcs-cpp

I have now got an audio pass-through program (2 different ones, actually) on my Teensy 3.2 and confirmed what I feared:

Experiment #1: Port the DC removal code to run on my PC and plot the output of the detected DC:
The result is ugly. Significant noise and sub-harmonic components show up in the plotted data. Also compared same data, same output and synthesis method to my IIR DC removal filter, and the IIR was clean, as expectd. This was simply to confirm my fixed-point synthesis and output was ok. Here attached images:
Here you can really see the aliasing ( 44100/128 - 277 = 68 Hz). Count the cycles and you will see this is about right -- I count about 9 cycles in 6000 samples, 44100/(6k/9) = 66 Hz, pretty close for eye-ball work.
DC_Remover_Output_Noise_277Hz_5k_to_35k.png

Just a few other conditions
DC_Remover_Output_Noise_440Hz_5k_to_35k.png

DC_Remover_Output_Noise_7768_to_57768.png



Experiment #2:
Do ADC to DAC pass-through using the ADC library, transferring ADC directly to DAC on a timed interrupt:
The result is acceptable audio quality matching my expectations for the setup and wiring job.

Experiment #3: ADC to DAC pass-through using the Audio library:
The result is ugly. The distortion is so bad it makes me wonder if anybody has even tried using the built-in ADC with the audio library yet?

Experiment #4: Implement a first-order IIR high-pass filter similar to what I presented in the first post and compare:
[EDIT] Done and works as expected. Sounds much better. Pull request submitted.

I added the Rockbox FRACMUL_SHL(int32_t x, int32_t y, int z) ASM implementation to utility/dspinst.h because it will be useful when coding & changing between arbitrary precision.

The following is the basis for what was put into the code for the pull request.

Code:
//
// dc_remover.c
//
// A test program to output text to be re-directed into a text file and/or
// copy/pasted into a spreadsheet or other plotting program.
//
// Demonstrates a fixed-point implementation of a
// first-order high-pass IIR Filter
//
// Compile with a simple 1-liner:
//    gcc -Wall dc_remover.c -o rmdc
// Typical Usage:
//   ./rmdc > plot_test.txt
// Then do whatever you like to plot or otherwise process the text file.
//

#include <stdio.h>
#include <stdlib.h>
#include <inttypes.h>

#define FS 44100

//
// FRACMUL implementations from Rockbox (rockbox.org) fracmul.h
// Of particular interest will be the ARM ASM implementation.
//
        /* Rockbox ASM implementations for the ARM specification */
    // /* Multiply two S.31 fractional integers and return the sign bit and the
    //  * 31 most significant bits of the result.
    //  */
    // static inline int32_t FRACMUL(int32_t x, int32_t y)
    // {
    //     int32_t t, t2;
    //     asm ("smull    %[t], %[t2], %[a], %[b]\n\t"
    //          "mov      %[t2], %[t2], asl #1\n\t"
    //          "orr      %[t], %[t2], %[t], lsr #31\n\t"
    //          : [t] "=&r" (t), [t2] "=&r" (t2)
    //          : [a] "r" (x), [b] "r" (y));
    //     return t;
    // }
    //
    // /* Multiply two S.31 fractional integers, and return the 32 most significant
    //  * bits after a shift left by the constant z.
    //  */
    // static FORCE_INLINE int32_t FRACMUL_SHL(int32_t x, int32_t y, int z)
    // {
    //     int32_t t, t2;
    //     asm ("smull    %[t], %[t2], %[a], %[b]\n\t"
    //          "mov      %[t2], %[t2], asl %[c]\n\t"
    //          "orr      %[t], %[t2], %[t], lsr %[d]\n\t"
    //          : [t] "=&r" (t), [t2] "=&r" (t2)
    //          : [a] "r" (x), [b] "r" (y),
    //            [c] "Mr" ((z) + 1), [d] "Mr" (31 - (z)));
    //     return t;
    // }

    /* Rockbox generic implementations */
static inline int32_t FRACMUL(int32_t x, int32_t y)
{
    return (int32_t) (((int64_t)x * y) >> 31);
}

static inline int32_t FRACMUL_SHL(int32_t x, int32_t y, int z)
{
    return (int32_t) (((int64_t)x * y) >> (31 - z));
}

//
// DC Removal Filter:
//  Takes an unsigned 16-bit integer (short) and returns a signed 32-bit integer
//  AC-centered at 0.
//
//  This filter implementation promotes fixed-point UNITY to (1<<20),
//  minimizing loss of accuracy.
//  Consequently the return value is in precision format S12.19
//  The final value should be rotated right 4 bits to get to S16.15
//  if it is to be cast back into a signed 16-bit integer (16-bit)
//
// 1-pole digital high-pass filter implementation
//   y = a*(x[n] - x[n-1] + y[n-1])
// The coefficient "a" is as follows:
//  a = UNITY*e^(-2*pi*fc/fs)
//
// For a sampling rate of 44100 Hz, a is selected according to 2^11
//  fc = 2 Hz
//  a = 1048300
// For a sampling rate of 96000 Hz,
//  fc=-ln(a)*fs/(2*pi)
//  fc = 4.35 Hz
// For a sampling rate of 192 kHz
//  fc = 8.7 Hz
// ^The selected coefficient should result in acceptable low-frequency performance
// for all imaginable sampling rates.
//

static int32_t hpf_x1, hpf_y1;  // HP Filter state variables
static int32_t a = 1048300;     // HP Filter coefficient : 2Hz @ 44.1k Hz

inline int32_t iir_hpf_1p(uint16_t x)
{
    int32_t x0 = ( ((int32_t) x) << 4);
    int32_t acc = x0;
    acc += hpf_y1;
    acc -= hpf_x1;
    hpf_y1 = FRACMUL_SHL(acc, a, 11);
    hpf_x1 = x0;
    return acc;
}

//
// Test routine to validate performance as expected.
//

int main()
{
    float T = 1.0/FS;
    float t = 0.0;

    int i = 0;
    unsigned short int xn = 28000;
    int32_t y = 0;
    int16_t ys = 0;
    hpf_x1 = 0;
    hpf_y1 = 0;

    for (i = 0; i < FS; i++)
    {
        y = iir_hpf_1p(xn);
        // Restore to 16 bit range, saturate and verify with cast into ys
        y = FRACMUL_SHL(y, (1<<20), 7);
        if(y > 32767) y = 32767; //Saturate pos
        else if(y < -32767) y = -32767; //Saturate pos

        ys = (int16_t) y;
        //apply step responses
        if( (i % (FS/50)) == 0)
        {
            if(xn == 10000)
                xn = 58000;
            else
                xn = 10000;
        }

        if(i%10 == 0) //Downsample the output to reduce amount of data to be plotted
            printf("%f\t%d\t%d\n", t, xn, ys);

        t += T;
    }

    return 0;
}

Looking at the datasheets, I would expect that comparable audio quality can be had from the built-in ADC as what is had from the SGTL5000.

I also have a plan to output PWM and DAC simultaneously, putting the MSB into the DAC and the LSB into the PWM, then sum the output through a 1:2^12 resistor divider followed by a 3rd-order butterworth filter. I will also have to make a library object to split the MSB and LSB from the 16-bit data, shift the LSB up to 16-bits and maybe even dither it.

With that strategy I'm expecting to get quality roughly comparable to the SGTL5000 (we will see).
 

Attachments

  • DC_Remover_Output_Noise_277Hz_5k_to_35k.png
    DC_Remover_Output_Noise_277Hz_5k_to_35k.png
    27.1 KB · Views: 310
Last edited:
Looped back DAC to ADC directly and analyzed return signal using USB interface and Audacity.

EXISTING DC REMOVAL

The step response and spectral plot make it more evident how the existing algorithm is downsampling followed by a moving average filter. The spectral plots were generated by playing a 277 Hz sine tone into Teensy 3.2 configured as a USB sound card. The Audio was routed out the 12-bit DAC and directly wired to the ADC input.

The result was recorded from the Teensy 3.2 USB sound card input channel and analyzed in Audacity.
FFT_Teensy_AudioLib_DC_Removal.png
FFT of returned spectrum makes the aliased (subharmonic) very obvious. This is coming from the real Teensy and not a simulation of ported code.
Transient_Trail_Teensy_AudioLib_DC_Removal.png
At the end of sine wave transmission this is zoomed to display the tail-end of the transient response of the existing DC Removal code.
Impulse_Response_Teensy_AudioLib_DC_Removal.png
This is the step response of the existing DC Removal code. The resampling is realized by integrating the input signal over each flat section of the stair step. The moving-average filter is applied to the sub-sampled signal. The result on the audio is to add in aliasing (and it sounds really bad, particularly when playing chords on a guitar, or music play-back).

Suggested Implementation with IIR High-Pass Filter
After implementing a high-pass filter at 2 Hz cut-off the following outputs were generated. The -62 dB hump near DC is due to numerical error in the filter implementation. For me it is acceptable but maybe worth a little more time to drop that below the noise floor.
FFT_HPF_IIR.png
Not sure why the spectrum looks more "dense" above 1 kHz, but the harmonic distortion is lower and the 68 Hz aliased term is gone. The noise floor runs closer to -70 dB (consistent with the 12-bit DAC).
Step_Response_HPF_IIR.png
Step response of the proposed filter shows the expected smooth exponential decay.
 
Last edited:
Updated to higher precision processing to take better advantage of the 64-bit accumulator. Here is what I consider a final product:
FFT_HPF_IIRq31.png
 
Back
Top