Checking ADC noise

Status
Not open for further replies.
Here is some real, non-shifted data I just took. Intuitively (which may be invalid), I'd expect the correct value to be a little less than 1304. Which is what the median with dither produces.

Who knows, there could be other algorithms that do better - like a mean after outliers have been removed.

Histogram:

1290 0
1291 1
1292 0
1293 1
1294 1
1295 0
1296 1
1297 0
1298 2
1299 0
1300 0
1301 0
1302 5
1303 125
1304 370
1305 50
1306 8
1307 0
1308 2
1309 0
1310 1
1311 0
1312 0
1313 0
1314 1
1315 0
1316 0
1317 1
1318 1
1319 0

Mean: 1304.06
Median: 1304
Median with dither: 1303.87
Mean from 1302 to 1306: 1303.75
 
I didn't include the full histogram, but in doing another test, indeed, the mean was much closer to the dithered median.

Does anyone have a conclusion - median is better than mean for oversampling or it's just misleading analysis?
 
Last edited:
Thanks Jnor!

For a measure of DC value, I would definitely choose Median.

You might get a more realistic value by averaging the centre 3, 4, 5, bins or so.

But excluding those single count bins that a deviate so much looks like a good idea to me. Those counts orininate form some form of interference IMHO and therefore do not add information about the DC voltage at the ADC input.
 
I updated the test code I posted yesterday based on jonr's post. This version is set up to use the 16-bit setting on the T3.6 ADC, so no dithering is involved. Once again, I used a NIMH cell with 1K in series. I tested with and without 0.1uF from ADC pin to ground---which seemed to make little difference.

The sampling scheme is the same: Oversample 32 times and compute a sample value from either the median of 32, or the mean of 32.

2000 of the oversampled values are collected and some statistics displayed. I also added the capability to do a dummy adc conversion before the 32 oversamples. With my signal source it made no difference

Sample output:
Code:
 Avg: 26036.15  P-P :   55.84 St.Dev:   6.368    Mean(32) 
 Avg: 26034.83  P-P :   68.00 St.Dev:   9.664    Median(32)

 Avg: 26036.67  P-P :   53.31 St.Dev:   6.493    Mean(32) 
 Avg: 26035.34  P-P :   63.00 St.Dev:  10.034    Median(32)

 Avg: 26036.63  P-P :   54.31 St.Dev:   6.605    Mean(32) 
 Avg: 26035.41  P-P :   65.00 St.Dev:  10.091    Median(32)

 Avg: 26035.59  P-P :   51.94 St.Dev:   5.919    Mean(32) 
 Avg: 26034.47  P-P :   64.00 St.Dev:   9.120    Median(32)

 Avg: 26036.62  P-P :   48.25 St.Dev:   6.479    Mean(32) 
 Avg: 26035.23  P-P :   63.00 St.Dev:   9.897    Median(32)

 Avg: 26036.26  P-P :   50.94 St.Dev:   6.266    Mean(32) 
 Avg: 26035.00  P-P :   71.00 St.Dev:   9.554    Median(32)

 Avg: 26035.11  P-P :   53.06 St.Dev:   5.959    Mean(32) 
 Avg: 26034.19  P-P :   63.00 St.Dev:   8.695    Median(32)

 Avg: 26035.28  P-P :   43.22 St.Dev:   5.998    Mean(32) 
 Avg: 26034.20  P-P :   58.00 St.Dev:   8.923    Median(32)

 Avg: 26035.20  P-P :   47.28 St.Dev:   6.226    Mean(32) 
 Avg: 26034.26  P-P :   63.00 St.Dev:   8.977    Median(32)

 Avg: 26035.64  P-P :   53.44 St.Dev:   6.148    Mean(32) 
 Avg: 26034.69  P-P :   63.00 St.Dev:   9.219    Median(32)

 Avg: 26036.09  P-P :   52.66 St.Dev:   5.976    Mean(32) 
 Avg: 26034.94  P-P :   59.00 St.Dev:   8.826    Median(32)

 Avg: 26034.96  P-P :   45.53 St.Dev:   5.962    Mean(32) 
 Avg: 26034.00  P-P :   60.00 St.Dev:   8.693    Median(32)

 Avg: 26035.83  P-P :   53.25 St.Dev:   5.909    Mean(32) D
 Avg: 26034.70  P-P :   66.00 St.Dev:   8.921    Median(32)

 Avg: 26036.94  P-P :   45.84 St.Dev:   6.350    Mean(32) D
 Avg: 26035.59  P-P :   64.00 St.Dev:   9.595    Median(32)

 Avg: 26036.48  P-P :   52.81 St.Dev:   6.315    Mean(32) D
 Avg: 26035.32  P-P :   68.00 St.Dev:   9.433    Median(32)

 Avg: 26037.01  P-P :   53.81 St.Dev:   6.329    Mean(32) D
 Avg: 26035.81  P-P :   68.00 St.Dev:   9.602    Median(32)

the D at the end of the line indicates when a dummy conversion was used.

I saved 32000 of the raw data samples before the oversampling so I could look at the raw data histogram.

Code:
 Avg: 26034.32  P-P :   48.09 St.Dev:   5.258    Mean(32) D
 Avg: 26033.47  P-P :   57.00 St.Dev:   7.360    Median(32)

Mean(32) method Histogram Data
  26012:      1      0      0      1      2      0      0      7      6      7 
  26022:      6     17     27     35     49     55     69    101    103    137 
  26032:    174    154    168    153    137    151    106     81     50     57 
  26042:     57     27     19     11      9      8      2      5      1      2 
  26052:      1      1      0      1      0      0      1      0      1 
Median(32) method Histogram Data
  26007:      3      0      0      0      1      1      2      0      6      6 
  26017:      6     14     15     20     20     35     32     62     56     64 
  26027:     83     90     72     76    105    120    118    118     98    114 
  26037:    110     90     60     62     68     59     47     33     27     26 
  26047:     19     17     11     11      5      5      5      2      2      1 
  26057:      0      0      1      0      1      0      0      1 
First 32000 Raw values Histogram Data
  25858:      0      0      0      0      0      0      0      0      0      0 
  25868:      0      0      0      0      0      0      0      0      0      0 
  25878:      0      0      0      0      0      0      0      0      0      0 
  25888:      0      0      0      0      1      0      0      0      0      0 
  25898:      0      0      0      0      0      0      0      0      0      0 
  25908:      0      0      0      0      0      2      0      0      0      0 
  25918:      0      1      0      0      1      0      0      1      0      2 
  25928:      0      0      1      1      0      0      1      1      1      3 
  25938:      0      2      0      2      1      1      1      1      1      2 
  25948:      4      2      4      3      4      7      3      5      4      2 
  25958:      6      7     14     11     17     26     15      9     13     35 
  25968:     33     31     30     48     42     44     49     57     65     58 
  25978:     67     92     83     84     81    114    129    161    119    178 
  25988:    155    155    181    168    219    210    217    258     90    158 
  25998:    183    236    337    267    276    317    299    332    342    296 
  26008:    342    343    294    190    240    203    240    244    357    372 
  26018:    379    354    416    358    356    345    443    373    364    355 
  26028:    426    338    319    351    415    373    368    360    392    344 
  26038:    356    304    396    387    333    476    322    295    324    307 
  26048:    424    355    315    372    380    345    363    310    439    338 
  26058:    373    361    390    334    308    289    365    327    317    332 
  26068:    301    292    275    238    255    271    226    201    135    161 
  26078:    167    166    230    199    167    154    169    133    115    104 
  26088:    132     76     68     65     78     46     39     29     40     30 
  26098:     15     23     17     10     13     10     10      6      5      6 
  26108:      4      2      3      3      1      3      1      1      2      2 
  26118:      2      2      2      3      0      1      3      0      0      1 
  26128:      0      1      1      1      1      0      2      1      0      0 
  26138:      1      2      1      0      2      0      0      0      0      0 
  26148:      0      1      0      1      0      0      1      0      0      0 
  26158:      0      0      0      0      0      0      0      0      0      0 
  26168:      0      0      0      0      0      0      0      0      0      0 
  26178:      0      0      0      0      0      0      0      1

One thing that stood out is that the sample averages from the mean(32) method are higher than those from the median(32) by about 1 count. I've yet to decide whether that's a problem in my calculations or an intrinsic difference based on the sample distribution. In this version of the program both the mean(32) and median(32) statistics are derived from the same oversampled data in one pass.

Collecting and saving one oversampled value takes about 200microseconds, so this collection will run out of steam at about 4K Samples/second on at T3.6 at 180MHz.

Here's the source code:
Code:
// Analog input test for Teensy 4.X   Oct 4 2012 J.Beale
// Hacked on by JonR (probably the source of any problems)
// Setup: https://picasaweb.google.com/109928236040342205185/Electronics#5795546092126071650
// Posted to PJRC form 8/20/20
// Modified by mborgerson 8/20/20
// Changed for T3.6 and 16-bit input 8/21/2020
// Switched to collecting samples and after-the-fact statistics
// Added histogram display of samples and raw data

#define VREF (3.2444)         // ADC reference voltage (= power supply)
#define VINPUT (1.04020)      // ADC input voltage from resistive divider to VREF
#define ADCMAX (65536)        // maximum possible reading from ADC
#define EXPECTED (ADCMAX*(VINPUT/VREF))    // expected ADC reading

#define OS     32            // how many times to oversample 
#define OFFSET  0 // -149.76  // calibrate out a fixed offset
#define num 1                 // ADC internal oversampling

const int analogInPin = A8;   // Analog input pin
const int ledpin = 13;

#define MAXBINS 1000
uint32_t histobins[MAXBINS];

#define SAMPLES 2000         // how many samples to combine for pp, std.dev statistics
#define RAWSAMPLEMAX 32000
float mnsamples[SAMPLES], mdsamples[SAMPLES];
float rawsamples[RAWSAMPLEMAX];
uint16_t rawmin,rawmax;
#define SAMPLERATE  1000
bool histoflag = false;
bool excelformat = false;
bool dummyconvert = false;

elapsedMicros sampletimer;
#define  LEDON digitalWriteFast(ledpin, HIGH);
#define  LEDOFF digitalWriteFast(ledpin, LOW);

void setup() {    // ==============================================================
  pinMode(analogInPin, INPUT_DISABLE); // because I don't know default at startup
  pinMode(ledpin, OUTPUT);
  analogReadAveraging(num);
  analogReadRes(16);  // Teensy 3.6: set ADC resolution to this many bits
  // on T4.X, Default Arduino AnalogRead(pin) takes about 6uSec---the slowest mode!
  Serial.begin(115200);      // baud rate is ignored with Teensy USB ACM i/o
  delay(1000);
  Serial.printf("# Teensy 3.6 16-bit ADC test start: %dx%d oversampling \n\n", OS, num);
  delay(1000);
  srand(micros()); // use unix time as random seed
  Serial.println();

} // ==== end setup() ===========


// for T3.6 with 16-bit ADC, samples are float
int sort_descD(const void *cmp1, const void *cmp2)
{
  // Need to cast the void * to uint16_t
  float a = *((float *)cmp1);
  float b = *((float *)cmp2);
  // The comparison
  return a > b ? -1 : (a < b ? 1 : 0);
}


// Added fixed timing for oversampled  acquisition
void loop() {  // ================================================================

  float datAvg1, datAvg2, pkpk1, pkpk2,std1,std2;
  float smin1,smax1, smin2,smax2;
  float ctimesum;
  uint32_t startctime;
  uint16_t adval;
  char ch;
  float x1,x2, ovsum;
  uint16_t rawidx;

  float array[OS];  // used to qsort 32 oversampled adc values
  ctimesum = 0; // for accumulating sum of conversion and calculation times
  sampletimer = 0;  // reset the sampling timer
  rawidx = 0;  rawmin = 65535;  rawmax = 0;
  for (int i = 0; i < SAMPLES; i++) {
    // wait until time for next sample
    while (sampletimer < (1000000 / SAMPLERATE));
    sampletimer = 0;  // reset the sampling timer
    startctime = micros();
    ovsum = 0;

    // do a first conversion to remove first-sample bias on higher impedance source      
    if(dummyconvert) adval = analogRead(analogInPin);
    for (int j = 0; j < OS; ++j) {
      adval = analogRead(analogInPin);
      if(rawidx < RAWSAMPLEMAX){
        rawsamples[rawidx] = adval;
        rawidx++;
      }
      if(adval > rawmax) rawmax = adval;
      if(adval < rawmin) rawmin = adval;
      ovsum += adval;  // for mean(32) calculation
      array[j] = adval;  // for median(32) calculation
    } // end of oversampling

    x1 = (ovsum / OS);  // This does the mean(32) calculation
    qsort(array, OS, sizeof(array[0]), sort_descD);
    x2 = array[OS / 2];  // This is the median(32) calculation
    mnsamples[i] = x1;
    mdsamples[i] = x2;

    ctimesum += micros() - startctime;
  }// end of sampling for loop


  MeanEtc(mnsamples, SAMPLES, &datAvg1,&std1,&smin1, &smax1);
  pkpk1 = smax1-smin1;
  MeanEtc(mdsamples, SAMPLES, &datAvg2,&std2,&smin2, &smax2);
  pkpk2 = smax2-smin2;
//  Serial.printf("\n Avg. Conversion time: %6.2f uSec\n", ctimesum / SAMPLES);
//  Serial.printf(" Raw min %u   Raw max %u\n", rawmin, rawmax);
  
  if (excelformat) { // print tab separated for direct paste to Excel
    Serial.printf("%8.2f \t%8.2f \t", datAvg1, datAvg2);
    Serial.printf("% 8.2f \t% 8.2f \t", pkpk1, pkpk2);
    Serial.printf("%6.3f\t %6.3f\t ", std1, std2);
    if(dummyconvert) Serial.print("D");
    Serial.println();
  } else {  //verbose format on two lines with labels

    Serial.printf(" Avg: %8.2f ", datAvg1);
    Serial.printf(" P-P :%8.2f", pkpk1);
    Serial.printf(" St.Dev: %7.3f    Mean(32) ", std1);
    if(dummyconvert) Serial.println("D"); else Serial.println();

    Serial.printf(" Avg: %8.2f ", datAvg2);
    Serial.printf(" P-P :%8.2f", pkpk2);
    Serial.printf(" St.Dev: %7.3f    Median(32)\n\n", std2);
  }

  if(histoflag){
    binMedian(mnsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin1, (uint16_t)smax1+1);
    Serial.print("Mean(32) method ");
    ShowHisto(histobins, (uint16_t)smin1, (uint16_t)smax1+1);
    
    binMedian(mdsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin2, (uint16_t)smax2+1);
    Serial.print("Median(32) method ");
    ShowHisto(histobins, (uint16_t)smin2, (uint16_t)smax2+1);

    binMedian(rawsamples, RAWSAMPLEMAX, histobins, MAXBINS, (uint16_t)rawmin, (uint16_t)rawmax+1);
    Serial.print("First 32000 Raw values ");
    ShowHisto(histobins, (uint16_t)rawmin, (uint16_t)rawmax+1);

    histoflag = false;
    Serial.println("\nPress any key to continue.");
    while(!Serial.available());
    ch = Serial.read();
  }
  CheckUserInput();
  
} // end loop()  =====================================================

// Simple user interface to switch between mean and median of 32 samples
// and to turn dithering on and off
void CheckUserInput(void) {
  char ch;
  if (Serial.available()) {
    ch = Serial.read();
    if (ch == 'e') excelformat = !excelformat;
    if (ch == 'd')  dummyconvert  = !dummyconvert;
    if (ch == 'h') histoflag = true;
  }
}


// do the  easy stats mean, median, mode, max, min
// converted input array to float because oversampled mean has fractional part
void MeanEtc(float dat[], uint32_t nvals, float *mnptr, float *stptr, float *minp, float *maxp) {
  uint32_t i;
  float_t imin, imax, ival;
  float  var, fv, s1, s2, mean;
  double  sum, sumsq; // with 4 million samples, even 32-bit floats have rounding problems!
  // NOTE SUMSQ gets LARGE: suppose the values are ~4000 and you sum 100,000 of them then the sumsq
  //       is about (4x10^3) * (4x10^3)  * (1x10^5)   or 1.6x10^12 which is way too big for a uint32_t
  sum = 0.0;
  sumsq = 0.0;
  imin = 65535;
  imax = 0;
  // first get sum and calculate mean
  for (i = 0; i < nvals; i++) {
    ival = dat[i];
    if (ival < imin) imin = ival;
    if (ival > imax) imax = ival;
    sum += (float)ival;
    sumsq += (float)ival * (float)ival;
  }
  mean = sum / nvals;
  // now rcalculate variance in tw-pass algorithm to minimize roundoff error
  s1 = 0; s2 = 0;
  for (i = 0; i < nvals; i++) {
    fv = dat[i] - mean;
    s1 += (fv * fv );
    s2 += fv;
  }
  var = (s1 - s2 * s2 / nvals) / (nvals - 1);
  *mnptr = mean;
  //Serial.printf("Variance: %8.3ef  s1: %8.3e  s2^2/n: %8.3e\n", var, s1, s2*s2/nvals);
  *stptr = sqrt(var);
  *minp = imin;
  *maxp = imax;
}


// Accumulate bin counts for histogram  data should be in range 0..65535
// takes about 140mSec to find median of 100,000 records
uint16_t binMedian(float dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
  uint32_t i, bsum, maxbinval;
  uint16_t val;

  if((imax-imin) > numbins){
    Serial.printf("Data range too high for %u bins\n", MAXBINS);
    return 0;
  }
  if(imax<=imin){
    Serial.printf("Max <= Min\n");
    return 0;
  }

  // empty the bins--they could be on the stack and have non-zero values
  memset(bins, 0, numbins * sizeof(uint32_t));
  //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
  //   delay(5);

  //maxbinval = 0;
  for (i = 0; i < numsamp; i++) {
    val = (uint16_t)dat[i]+0.5; // round data to integer
    if ((val-imin) < numbins) { // prevent writing past end of bins array
      bins[val-imin]++;
 //     if(bins[val-imin > maxbinval){
 //       *modeptr = val;
 //       maxbinval = bins[val];
 //    }
    }

  } // bins are filled
  i = 0; bsum = 0;
  do {
    bsum += bins[i];
    i++;
  }  while (bsum < numsamp / 2);

  return i - 1;
}

// with new binMedian algorithm, lowest sample value is in bin[0]
void ShowHisto(uint32_t bins[], uint16_t imin, uint16_t imax) {
  uint16_t i, lnum;
  lnum = 0;
  Serial.print("Histogram Data");
  for (i = 0; i < (imax-imin); i++) {
    if ((lnum % 10) == 0) Serial.printf("\n % 4d: ", i+imin);
    lnum++;
    Serial.printf(" % 5lu ", bins[i]);
  }
  Serial.println();
}
 
The difference between the mean and median algorithms gets worse when I collect from a voltage divider source consisting of two 50K resistors.
Code:
 Avg: 32719.75  P-P :   92.62 St.Dev:  13.380    Mean(32) 
 Avg: 32716.36  P-P :  143.00 St.Dev:  21.539    Median(32)

Mean(32) method Histogram Data
  32677:      1      0      2      0      0      2      1      1      1      2 
  32687:      0      5      6      3     11      5      8     12     10     15 
  32697:     13     11     22     19     29     31     29     33     27     47 
  32707:     48     34     53     41     70     45     50     50     58     53 
  32717:     51     55     65     47     74     51     60     52     55     53 
  32727:     50     54     48     44     36     46     38     33     19     24 
  32737:     23     12     21     19     16     13     16     12      8     11 
  32747:     11      3      4      4      6      3      4      3      1      2 
  32757:      0      1      0      1      0      0      0      0      1      0 
  32767:      1      0      0      1 
Median(32) method Histogram Data
  32635:      1      0      0      0      0      1      0      0      0      0 
  32645:      0      0      0      0      0      0      0      0      0      0 
  32655:      1      1      0      0      1      0      0      0      2      0 
  32665:      1      0      3      0      1      3      3      3      3      2 
  32675:      2      3      8     10      4     13      6     15     14     11 
  32685:     16     21     27     19     20     27     27     28     17     33 
  32695:     18     25     17     31     32     35     37     21     25     26 
  32705:     19     43     36     34     35     40     22     31     26     38 
  32715:     35     29     36     40     34     37     32     50     37     38 
  32725:     45     39     36     27     23     43     33     32     22     10 
  32735:     15     19     19     23     21     19     15     23     11     12 
  32745:     10     14      7     19     12      9      9     11      7     12 
  32755:     11      9      4      4      8      3      2      6      5      4 
  32765:      6      4      6      5      1      9      0      1      4      2 
  32775:      2      0      0      1 
First 32000 Raw values Histogram Data
  32459:      0      0      0      0      0      0      0      0      0      0 
  32469:      0      0      0      0      0      0      0      0      0      0 
  32479:      0      0      0      0      0      0      0      0      0      0 
  32489:      0      0      0      0      0      0      0      0      0      0 
  32499:      0      0      1      0      0      0      0      1      1      0 
  32509:      1      1      1      1      0      1      0      1      2      0 
  32519:      0      0      0      1      1      3      1      2      0      1 
  32529:      3      2      2      2      2      1      3      2      2      0 
  32539:      3      3      8      2      9      4      5      6      9      5 
  32549:      7      4      5     11      5     12     14     14     12     11 
  32559:     18      5      8     19     17      7     20     19     17     15 
  32569:     21     18     19     13     29     25     29     24     29     43 
  32579:     34     28     38     30     42     30     30     55     49     32 
  32589:     34     63     50     45     35     57     59     53     64     64 
  32599:     60     59     47     68     68     46     62     52     59     61 
  32609:     53     92     82     68     66     89     71     76     62    114 
  32619:     83     90     72    145    109     90     84    121    101    101 
  32629:    114    146    119    120     94    159    107    108    156    119 
  32639:    110    126     83    172    121    116    135    139    139    116 
  32649:    121    148    150    133    135    174    155    125    134    158 
  32659:    151    130    120    170    135    132    123    182    153    109 
  32669:    135     85    111    111    112    170    129    132    125    162 
  32679:    144    126    114    185    143    133    121    178    147    112 
  32689:    102    186    125    131    119    147    119    116    103    159 
  32699:    147    122    175    115    100    115    109    170    124    110 
  32709:    106    156    107    120     98    145    134    117    132    148 
  32719:    119    111    113    163    138    141    139    136    122    125 
  32729:    105    148    108    108    101     72     76    113    107    155 
  32739:    125    110    128    155    126    102    109    154    132     94 
  32749:    119    184    120    115    122    161    149    135    121    158 
  32759:    141    131    124    193    159    159    231    137    141    123 
  32769:    129    178    145    147    168    161    145    153    118    172 
  32779:    154    138    129    181    136    132    142    164    144    132 
  32789:    157    153    125    135    109    155    116    105    101     89 
  32799:    117    123    114    147    113    116    104    114    104    130 
  32809:     97    158    116    103    105    147    106     87     89    151 
  32819:    115     96     97    121     82     87     68    109     91     87 
  32829:     91     73     69     73     55     87     77     65     60     65 
  32839:     72     72     41     69     63     52     54     57     43     37 
  32849:     43     48     41     46     39     38     39     33     34     27 
  32859:     37     25     26     13     23     22     17     33     26     18 
  32869:     18     18     19     15      7     16     15     17      8      7 
  32879:     13     14      8     12     15      8     10      8      6      5 
  32889:      6      7      7      5      4      5      7      1      8      3 
  32899:      5      2      3      3      2      2      2      1      3      1 
  32909:      0      3      1      1      0      2      1      0      2      1 
  32919:      2      0      1      2      0      0      0      0      2      0 
  32929:      0      0      0      0      0      1      0      0      0      2 
  32939:      0      0      0      0      0      0      0      0      0
 
That latest software won't run on my teensy 3.1 due to memory, but with my sw version and hardware, median slightly outperforms mean.
 
That latest software won't run on my teensy 3.1 due to memory, but with my sw version and hardware, median slightly outperforms mean.

OOPS! I copied the code before I changed the data type for rawsamples from float to uint16_t. The following code should work, and I updated some of the histogram routines to make sure the max and min values were displayed.

Code:
// Analog input test for Teensy 4.X   Oct 4 2012 J.Beale
// Hacked on by JonR (probably the source of any problems)
// Setup: https://picasaweb.google.com/109928236040342205185/Electronics#5795546092126071650
// Posted to PJRC form 8/20/20
// Modified by mborgerson 8/20/20
// Changed for T3.6 and 16-bit input 8/21/2020
// Switched to collecting samples and after-the-fact statistics
// Added histogram display of samples and raw data

#define VREF (3.2444)         // ADC reference voltage (= power supply)
#define VINPUT (1.04020)      // ADC input voltage from resistive divider to VREF
#define ADCMAX (65536)        // maximum possible reading from ADC
#define EXPECTED (ADCMAX*(VINPUT/VREF))    // expected ADC reading

#define OS     32            // how many times to oversample 
#define OFFSET  0 // -149.76  // calibrate out a fixed offset
#define num 1                 // ADC internal oversampling

const int analogInPin = A8;   // Analog input pin
const int ledpin = 13;

#define MAXBINS 1000
uint32_t histobins[MAXBINS];

#define SAMPLES 2000         // how many samples to combine for pp, std.dev statistics
#define RAWSAMPLEMAX 64000
float mnsamples[SAMPLES], mdsamples[SAMPLES];
uint16_t rawsamples[RAWSAMPLEMAX];
uint16_t rawmin,rawmax;
#define SAMPLERATE  500
bool histoflag = false;
bool excelformat = false;
bool dummyconvert = false;

elapsedMicros sampletimer;
#define  LEDON digitalWriteFast(ledpin, HIGH);
#define  LEDOFF digitalWriteFast(ledpin, LOW);

void setup() {    // ==============================================================
  pinMode(analogInPin, INPUT_DISABLE); // because I don't know default at startup
  pinMode(ledpin, OUTPUT);
  analogReadAveraging(4);
  analogReadRes(16);  // Teensy 3.6: set ADC resolution to this many bits
  // on T4.X, Default Arduino AnalogRead(pin) takes about 6uSec---the slowest mode!
  Serial.begin(115200);      // baud rate is ignored with Teensy USB ACM i/o
  delay(1000);
  Serial.printf("# Teensy 3.6 16-bit ADC test start: %dx%d oversampling \n\n", OS, num);
  delay(1000);
  srand(micros()); // use unix time as random seed
  Serial.println();

} // ==== end setup() ===========


// for T3.6 with 16-bit ADC, samples are float
int sort_descD(const void *cmp1, const void *cmp2)
{
  // Need to cast the void * to uint16_t
  float a = *((float *)cmp1);
  float b = *((float *)cmp2);
  // The comparison
  return a > b ? -1 : (a < b ? 1 : 0);
}


// Added fixed timing for oversampled  acquisition
void loop() {  // ================================================================

  float datAvg1, datAvg2, pkpk1, pkpk2,std1,std2;
  float smin1,smax1, smin2,smax2;
  float ctimesum;
  uint32_t startctime;
  uint16_t adval;
  char ch;
  float x1,x2, ovsum;
  uint16_t rawidx, rawmed;

  float array[OS];  // used to qsort 32 oversampled adc values
  ctimesum = 0; // for accumulating sum of conversion and calculation times
  sampletimer = 0;  // reset the sampling timer
  rawidx = 0;  rawmin = 65535;  rawmax = 0;
  for (int i = 0; i < SAMPLES; i++) {
    // wait until time for next sample
    while (sampletimer < (1000000 / SAMPLERATE));
    sampletimer = 0;  // reset the sampling timer
    startctime = micros();
    ovsum = 0;

    // do a first conversion to remove first-sample bias on higher impedance source      
    if(dummyconvert) adval = analogRead(analogInPin);
    for (int j = 0; j < OS; ++j) {
      adval = analogRead(analogInPin);
      if(rawidx < RAWSAMPLEMAX){
        rawsamples[rawidx] = adval;
        rawidx++;
      }
      if(adval > rawmax) rawmax = adval;
      if(adval < rawmin) rawmin = adval;
      ovsum += adval;  // for mean(32) calculation
      array[j] = (float)adval;  // for median(32) calculation
    } // end of oversampling

    x1 = (ovsum / OS);  // This does the mean(32) calculation
    qsort(array, OS, sizeof(array[0]), sort_descD);
    x2 = array[OS / 2];  // This is the median(32) calculation
    mnsamples[i] = x1;
    mdsamples[i] = x2;

    ctimesum += micros() - startctime;
  }// end of sampling for loop


  MeanEtc(mnsamples, SAMPLES, &datAvg1,&std1,&smin1, &smax1);
  pkpk1 = smax1-smin1;
  MeanEtc(mdsamples, SAMPLES, &datAvg2,&std2,&smin2, &smax2);
  pkpk2 = smax2-smin2;
//  Serial.printf("\n Avg. Conversion time: %6.2f uSec\n", ctimesum / SAMPLES);
//  Serial.printf(" Raw min %u   Raw max %u\n", rawmin, rawmax);
  
  if (excelformat) { // print tab separated for direct paste to Excel
    Serial.printf("%8.2f \t%8.2f \t", datAvg1, datAvg2);
    Serial.printf("% 8.2f \t% 8.2f \t", pkpk1, pkpk2);
    Serial.printf("%6.3f\t %6.3f\t ", std1, std2);
    if(dummyconvert) Serial.print("D");
    Serial.println();
  } else {  //verbose format on two lines with labels

    Serial.printf(" Avg: %8.2f ", datAvg1);
    Serial.printf(" P-P :%8.2f", pkpk1);
    Serial.printf(" St.Dev: %7.3f    Mean(32) ", std1);
    if(dummyconvert) Serial.println("D"); else Serial.println();

    Serial.printf(" Avg: %8.2f ", datAvg2);
    Serial.printf(" P-P :%8.2f", pkpk2);
    Serial.printf(" St.Dev: %7.3f    Median(32)\n\n", std2);
  }

  if(histoflag){
    binMedianF(mnsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin1, (uint16_t)smax1);
    Serial.print("Mean(32) method ");
    ShowHisto(histobins, (uint16_t)smin1, (uint16_t)smax1);
    
    binMedianF(mdsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin2, (uint16_t)smax2);
    Serial.print("Median(32) method ");
    ShowHisto(histobins, (uint16_t)smin2, (uint16_t)smax2);

    rawmed = binMedianU(rawsamples, RAWSAMPLEMAX, histobins, MAXBINS, (uint16_t)rawmin, (uint16_t)rawmax);
    Serial.printf("Median of Raw values: %u   ",rawmed);
    ShowHisto(histobins, (uint16_t)rawmin, (uint16_t)rawmax);

    histoflag = false;
    Serial.println("\nPress any key to continue.");
    while(!Serial.available());
    ch = Serial.read();
  }
  CheckUserInput();
  
} // end loop()  =====================================================

// Simple user interface to switch between mean and median of 32 samples
// and to turn dithering on and off
void CheckUserInput(void) {
  char ch;
  if (Serial.available()) {
    ch = Serial.read();
    if (ch == 'e') excelformat = !excelformat;
    if (ch == 'd')  dummyconvert  = !dummyconvert;
    if (ch == 'h') histoflag = true;
  }
}


// do the  easy stats mean, median, mode, max, min
// converted input array to float because oversampled mean has fractional part
void MeanEtc(float dat[], uint32_t nvals, float *mnptr, float *stptr, float *minp, float *maxp) {
  uint32_t i;
  float_t imin, imax, ival;
  float  var, fv, s1, s2, mean;
  double  sum, sumsq; // with 4 million samples, even 32-bit floats have rounding problems!
  // NOTE SUMSQ gets LARGE: suppose the values are ~4000 and you sum 100,000 of them then the sumsq
  //       is about (4x10^3) * (4x10^3)  * (1x10^5)   or 1.6x10^12 which is way too big for a uint32_t
  sum = 0.0;
  sumsq = 0.0;
  imin = 65535;
  imax = 0;
  // first get sum and calculate mean
  for (i = 0; i < nvals; i++) {
    ival = dat[i];
    if (ival < imin) imin = ival;
    if (ival > imax) imax = ival;
    sum += (float)ival;
    sumsq += (float)ival * (float)ival;
  }
  mean = sum / nvals;
  // now rcalculate variance in tw-pass algorithm to minimize roundoff error
  s1 = 0; s2 = 0;
  for (i = 0; i < nvals; i++) {
    fv = dat[i] - mean;
    s1 += (fv * fv );
    s2 += fv;
  }
  var = (s1 - s2 * s2 / nvals) / (nvals - 1);
  *mnptr = mean;
  //Serial.printf("Variance: %8.3ef  s1: %8.3e  s2^2/n: %8.3e\n", var, s1, s2*s2/nvals);
  *stptr = sqrt(var);
  *minp = imin;
  *maxp = imax;
}


// Accumulate bin counts for histogram  data should be in range 0..65535
// takes about 140mSec to find median of 100,000 records
// this version for uint16_t dat
uint16_t binMedianU(uint16_t dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
  uint32_t i, bsum, maxbinval;
  uint16_t val;

  if((imax-imin) > numbins){
    Serial.printf("Data range too high for %u bins\n", MAXBINS);
    return 0;
  }
  if(imax<=imin){
    Serial.printf("Max <= Min\n");
    return 0;
  }

  Serial.printf("min: %u   and max %u \n",imin, imax);
  // empty the bins--they could be on the stack and have non-zero values
  memset(bins, 0, numbins * sizeof(uint32_t));
  //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
  //   delay(5);

  //maxbinval = 0;
  for (i = 0; i < numsamp; i++) {
    val = dat[i]; 
   
    if ((val-imin) < numbins) { // prevent writing past end of bins array
      bins[val-imin]++;
    } else printf("Out of Bounds with %u   and min %u\n", val, imin);

  } // bins are filled
  i = 0; bsum = 0;
  do {
    bsum += bins[i];
    i++;
  }  while (bsum < numsamp / 2);

  return imin+ i - 1;
}


// Accumulate bin counts for histogram  data should be in range 0..65535
// takes about 140mSec to find median of 100,000 records
// this version for floating point dat
uint16_t binMedianF(float dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
  uint32_t i, bsum, maxbinval;
  uint16_t val;

  if((imax-imin) > numbins){
    Serial.printf("Data range too high for %u bins\n", MAXBINS);
    return 0;
  }
  if(imax<=imin){
    Serial.printf("Max <= Min\n");
    return 0;
  }

  Serial.printf("min: %u   and max %u \n",imin, imax);
  // empty the bins--they could be on the stack and have non-zero values
  memset(bins, 0, numbins * sizeof(uint32_t));
  //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
  //   delay(5);

  //maxbinval = 0;
  for (i = 0; i < numsamp; i++) {
    val = (uint16_t)dat[i]; // truncate data to integer
//    if(val == imin) Serial.printf( "min of %u at %u\n", val, i);
 //   if(val == imax-1) Serial.printf( "max of %u at %u\n", val, i);
  
    if ((val-imin) < numbins) { // prevent writing past end of bins array
      bins[val-imin]++;
    } else printf("Out of Bounds with %u   and min %u\n", val, imin);

  } // bins are filled
  i = 0; bsum = 0;
  do {
    bsum += bins[i];
    i++;
  }  while (bsum < numsamp / 2);

  return  imin+i - 1;
}


// with new binMedian algorithm, lowest sample value is in bin[bins]
void ShowHisto(uint32_t bins[], uint16_t imin, uint16_t imax) {
  uint16_t i, lnum;
  lnum = 0;
  Serial.print("Histogram Data");
  for (i = 0; i < (imax-imin+1); i++) {
    if ((lnum % 10) == 0) Serial.printf("\n % 4d: ", i+imin);
    lnum++;
    Serial.printf(" % 5lu ", bins[i]);
  }
  Serial.println();
}
 
Got it to run by reducing RAWSAMPLEMAX. A typical T3.1 result:

Avg: 21024.89 P-P : 23.81 St.Dev: 3.713 Mean(32)
Avg: 21025.63 P-P : 11.00 St.Dev: 2.505 Median(32)
 
Great work, but I am very curious how the ADC performs with a low impedance source, likr 10R or so. A real low noise stable, low impedance source..... cannt you repeat this with a much larger capacitor to gnd?
 
I am sure you can get St.Dev down to 1 with a larger oversampling number, like 128 or so. Then you would have 16bit performace.
 
I am sure you can get St.Dev down to 1 with a larger oversampling number, like 128 or so. Then you would have 16bit performace.

Depends what you mean by 16 bit performance - resolution perhaps, but there's integral non-linearity, and drift to consider, and
the circuit needs to be good enough too (precision voltage reference, ground loops avoided, etc). And this is DC performance only
rather than dynamic (much harder to get dynamic performance of course).
 
> Depends what you mean by 16 bit performance ...

Exactly. For some uses, a teensy can be similar to a good external 16 bit converter if you over sample by 4000x. In other ways, no amount of oversampling gets there.

In my tests, the T4 outperforms the T3. The latter introduces a big offset every time it re-calibrates. It probably is possible to write a lower noise calibration routine.
 
Depends what you mean by 16 bit performance - resolution perhaps, but there's integral non-linearity, and drift to consider, and
the circuit needs to be good enough too (precision voltage reference, ground loops avoided, etc). And this is DC performance only
rather than dynamic (much harder to get dynamic performance of course).

When I mean 16bit performance, I mean performance equivalent to a typical 16bit ADC IC. Once you get standard deviation of about 2^-15 FullScale, by averaging multiple sample, (e.g. 256 or so) you have the equivalent of a 16bit ADC IC. This has nothing to do with the dynamics of a signal, not with drift, nor with offset.
 
> Depends what you mean by 16 bit performance ...

Exactly. For some uses, a teensy can be similar to a good external 16 bit converter if you over sample by 4000x. In other ways, no amount of oversampling gets there.

In my tests, the T4 outperforms the T3. The latter introduces a big offset every time it re-calibrates. It probably is possible to write a lower noise calibration routine.

I see no problem with averaging 4000 samples, allthough 256 should be enough to get the StDev down to 2^-15 FS . I regulairy average traces of 40e3 samples, sampled at 200MHz with a 14bit ADC, 10e3..40e3 times to fish low level signals out of noisy signals.
 
Here is some "trimmed mean" code that often does better than either median or mean.

Code:
int compare_u16(const void *cmp1, const void *cmp2)
{
  // Need to cast the void * 
  const uint16_t a = *((uint16_t *)cmp1);
  const uint16_t b = *((uint16_t *)cmp2);
  return a > b ? -1 : (a < b ? 1 : 0);
}

// take mean of samples on each side of the median
double trimmed_mean(const uint16_t *array, const unsigned count, const unsigned n)
{
  qsort(array, count, sizeof(array[0]), compare_u16);

  unsigned sum = 0;
  unsigned n2 = 0;
  const int median = array[count / 2];

  const unsigned high = array[count/2 - 1];  // caution: decending sort
  const unsigned low = array[count/2 + 1];

  for (unsigned i = 0; i < count; ++i) {
    //Serial.printf("%u ",array[i]);
    if ((array[i] >= median - n) && (array[i] <= median + n)) {  // alternative method
    //if ((array[i] >= low) && (array[i] <= high)) {
      sum += array[i];
      ++n2;
    }
  } // for

  //Serial.printf(" med = %u, sum = %u, n = %u\n",median, sum,n2);

  return (double)sum / n2;
}
 
When I mean 16bit performance, I mean performance equivalent to a typical 16bit ADC IC. Once you get standard deviation of about 2^-15 FullScale, by averaging multiple sample, (e.g. 256 or so) you have the equivalent of a 16bit ADC IC. This has nothing to do with the dynamics of a signal, not with drift, nor with offset.

Not sure there's such a thing as a typical 16bit ADC. For instance audio class ADCs have very different performance
criteria than instrumentation class ADCs, and there are very different implementation methods for precision ADCs,
each with their own strengths/weaknesses - SAR and sigma/delta and flash techniques are all used as are many
proprietry hybrid approaches.

I think you are talking about DC accuracy or perhaps DC repeatability? Dual slope integrating ADCs are actually a pretty
good choice for this and offer more than 16 bits of true accuracy too. https://www.ti.com/lit/an/snoa597b/snoa597b.pdf

For varying signals this isn't realistic, so an SAR ADC might be the way to go, and a typical 16 bit SAR might be accurate
(or at least linear) to 14.5 bits or so with a really good voltage reference and attention to detail. You won't get anything
like that from the T4 built-in ADC, merely the illusion of accuracy (ie short-term repeatability). Same goes for any 10 bit
ADC without a good voltage reference. The 10 bit staircase is not going to be accurate or stable to 16 bits, that would
require a different process for making the die for one thing, with laser trimming and so forth.
 
> Depends what you mean by 16 bit performance ...

Exactly. For some uses, a teensy can be similar to a good external 16 bit converter if you over sample by 4000x. In other ways, no amount of oversampling gets there.

In my tests, the T4 outperforms the T3. The latter introduces a big offset every time it re-calibrates. It probably is possible to write a lower noise calibration routine.

If the errors are in the ADC, not in the input multiplexers, you could get better performance on the T3 by using two spare channels, one for a stable 2.5V reference and another for the same reference divided down to something near your lowest signal of interest. (Hopefully that lowest signal is well above ground so you don't have to worry about clipping.) At regular intervals, you could sample the two reference channels and use them for a two-point calibration curve to correct the signal on your sensor channel(s). If that works out, you can start worrying about the temperature stability of your sensors! ;-)
 
> At regular intervals, you could sample the two reference channels and use them for a two-point calibration ...

Even before every sample. Your thinking is similar to a lock-in amplifier, which can pull clean signals out of incredible amounts of noise - even when the noise is about the same (or lower) frequency as the signal.
 
Would be interesting to know if the analog comparator on the teensy could be used to implement a dual-slope integrating ADC that outperformed the built-in ADC (at low speeds).
 
I updated my test code with the following 'features':
1. You can switch between simulated normally-distributed data with the 'a' key.
2. The simulated data has a variable std deviation, which you can change: '4':4.0, '5':8,'6':16,'7':32.
3. You can vary the number of oversamples. '1':32, '2':128,'3':1024. Note that you can't maintain 1KHz at the higher oversample rates with the slow ADC settings. The simulated data generator is faster, though.

I built a stable low-noise voltage source with an LTC6655 2.5V reference chip that has been trying to escape my parts bin since 2014. The proto board connects with short connections to avoid noise pickup. See Photo:

VSource.jpg

The voltage source connects 2.5V to A9 and 1.0V to A8--the latter so I can test the ADC with the 1.2V internal reference.

Here are the results of some tests:
Code:
Simulated normally-distributed data oversampled 32 times, stats for 2000 samples.

 Avg: 19999.99  P-P :   10.28 St.Dev:   1.399    Mean(32) S
 Avg: 19999.70  P-P :   12.00 St.Dev:   1.777    Median(32)  ctime: 114.2

 Avg: 19999.94  P-P :    9.12 St.Dev:   1.391    Mean(32) S
 Avg: 19999.64  P-P :   13.00 St.Dev:   1.770    Median(32)  ctime: 114.2

min: 19995   and max 20004 
Mean(32) method Histogram Data
  19995:      3     36    129    320    514    531    342    105     18      2 
min: 19993   and max 20006 
Median(32) method Histogram Data
  19993:      1      2     13     63    155    270    415    462    314    213 
  20003:     70     20      1      1 
min: 19963   and max 20030 
Median of Raw values: 20000   Histogram Data
  19963:      2      0      1      0      3      1      1      2      4      8 
  19973:     13     24     27     45     61     75    115    157    193    255 
  19983:    316    416    564    722    887   1063   1237   1453   1631   1941 
  19993:   2132   2382   2643   2879   2990   3071   3184   3233   3110   3100 
  20003:   3017   2793   2620   2489   2170   1958   1680   1412   1237   1008 
  20013:    843    636    531    428    339    249    191    151    106     66 
  20023:     46     26     20     21     11      7      2      2 

Analog data from 2.5V LTC6655 voltage refrence.

 Avg: 49553.83  P-P :    6.19 St.Dev:   0.857    Mean(32) A
 Avg: 49559.06  P-P :    8.00 St.Dev:   1.030    Median(32)  ctime: 590.1

 Avg: 49553.85  P-P :    5.81 St.Dev:   0.854    Mean(32) A
 Avg: 49559.17  P-P :    6.00 St.Dev:   1.031    Median(32)  ctime: 590.1

min: 49551   and max 49557 
Mean(32) method Histogram Data
  49551:     29    291    815    678    170     16      1 
min: 49556   and max 49562 
Median(32) method Histogram Data
  49556:      2     90    416    747    569    153     23 
min: 49487   and max 49569 
Median of Raw values: 49559   Histogram Data
  49487:      1      8     14     49    120    185    271    341    378    263 
  49497:    186    108     48     20      6      2      0      2      7     19 
  49507:     52    129    214    330    387    346    231    158     78     30 
  49517:     12      4      8     23     48     76    144    213    230    267 
  49527:    249    257    185    130     93     58     42     59    101    170 
  49537:    242    285    309    274    245    185    176    206    252    341 
  49547:    371    447    484    496    610    666    793   1045   1448   2204 
  49557:   3625   5614   7447   8577   8076   6301   3858   1870    830    256 
  49567:     83     25      7

It's apparent that the simulated data has a nice normal distribution and the mean and median results are well within 1LSB of each other. The ADC results are NOT normally distributed and the mean and median method results are far from each other. I measured the 3.3V rail as 3.3065V. When I plug that number and 2.5002 V at the input, the theoretical number of counts the ADC should see is 49,554.8.
The mean method gives a result within 1LSB at 49553.85. The median method gives a number about 4 counts higher. The mean method shows about 6 counts P-P noise, (about 300 microVolts). The noise specification for the LTC6655 is 0.67PPM at 1KHz and even lower at the oversampling frequency due to the 10uF cap on the output. That low noise figure and the skewed distribution lead me to believe that the noise is in the ADC, not my voltage source.

Here's the source code so you can try it out with your own sources and sensors:
Code:
// Analog input test for Teensy 4.X   Oct 4 2012 J.Beale
// Hacked on by JonR (probably the source of any problems)
// Setup: https://picasaweb.google.com/109928236040342205185/Electronics#5795546092126071650
// Posted to PJRC form 8/20/20
// Modified by mborgerson 8/20/20
// Changed for T3.6 and 16-bit input 8/21/2020
// Switched to collecting samples and after-the-fact statistics
// Added histogram display of samples and raw data

#define VREF (3.2444)         // ADC reference voltage (= power supply)
#define VINPUT (1.04020)      // ADC input voltage from resistive divider to VREF
#define ADCMAX (65536)        // maximum possible reading from ADC
#define EXPECTED (ADCMAX*(VINPUT/VREF))    // expected ADC reading


#define OFFSET  0 // -149.76  // calibrate out a fixed offset
#define num 1                 // ADC internal oversampling

const int analogInPin = A9;   // Analog input pin
const int ledpin = 13;

#define MAXBINS 1000
uint32_t histobins[MAXBINS];

#define OSMAX 1024
/*  for later implementation of running mean
uint16_t rmdat[MAXOS];
uint16_t rmidx = 0;
uint32_t rmsum;
*********************************************/
float simStd = 8.0;

#define SAMPLES 2000         // how many samples to combine for pp, std.dev statistics
#define RAWSAMPLEMAX 64000

uint16_t OS = 32;
float mnsamples[SAMPLES], mdsamples[SAMPLES];
uint16_t rawsamples[RAWSAMPLEMAX];
uint16_t rawmin,rawmax;
#define SAMPLERATE  500
bool histoflag = false;
bool excelformat = false;
bool useanalog = false;

elapsedMicros sampletimer;
#define  LEDON digitalWriteFast(ledpin, HIGH);
#define  LEDOFF digitalWriteFast(ledpin, LOW);

void setup() {    // ==============================================================
  pinMode(analogInPin, INPUT_DISABLE); // because I don't know default at startup
  pinMode(ledpin, OUTPUT);
  analogReadAveraging(4);
  analogReadRes(16);  // Teensy 3.6: set ADC resolution to this many bits
  // on T4.X, Default Arduino AnalogRead(pin) takes about 6uSec---the slowest mode!
  Serial.begin(115200);      // baud rate is ignored with Teensy USB ACM i/o
  delay(1000);
  Serial.printf("# Teensy 3.6 16-bit ADC test start: %dx%d oversampling \n\n", OS, num);
  delay(1000);
  srand(micros()); // use unix time as random seed
  Serial.println();
  ARM_DEMCR |= ARM_DEMCR_TRCENA;
  ARM_DWT_CTRL |= ARM_DWT_CTRL_CYCCNTENA;

} // ==== end setup() ===========


// for T3.6 with 16-bit ADC, samples are float because mean and running mean return values with fractional part
int sort_descD(const void *cmp1, const void *cmp2)
{
  // Need to cast the void * to uint16_t
  float a = *((float *)cmp1);
  float b = *((float *)cmp2);
  // The comparison
  return a > b ? -1 : (a < b ? 1 : 0);
}

// Simple user interface to switch between mean and median of 32 samples
// and to turn dithering on and off
void CheckUserInput(void) {
  char ch;
  if (Serial.available()) {
    ch = Serial.read();
    if (ch == 'e') excelformat = !excelformat;
    if (ch == 'a') useanalog = !useanalog;
    if (ch == 'h') histoflag = true;
    if (ch == '1') OS = 32;
    if (ch == '2') OS = 128;
    if (ch == '3') OS = 1024;
    if (ch == '4') simStd = 4.0;
    if (ch == '5') simStd = 8.0;
    if (ch == '6') simStd = 16.0;
    if (ch == '7') simStd = 32.0;   
    srand(micros()); // reseed the RNG
  }
}

 float medarray[OSMAX];  // used to qsort oversampled adc values

//  uint32_t dwt;
 // dwt = ARM_DWT_CYCCNT;

// Added fixed timing for oversampled  acquisition
void loop() {  // ================================================================

  float datAvg1, datAvg2, pkpk1, pkpk2,std1,std2;
  float smin1,smax1, smin2,smax2;
  float ctimesum;
  uint32_t startctime;
  uint16_t adval;
  char ch;
  float x1,x2, ovsum;
  uint16_t rawidx, rawmed;

 
  ctimesum = 0; // for accumulating sum of conversion and calculation times
  sampletimer = 0;  // reset the sampling timer
  rawidx = 0;  rawmin = 65535;  rawmax = 0;
  for (int i = 0; i < SAMPLES; i++) {
    // wait until time for next sample
    while (sampletimer < (1000000 / SAMPLERATE));
    sampletimer = 0;  // reset the sampling timer
    startctime = micros();
    ovsum = 0;

    for (int j = 0; j < OS; ++j) {
      if(useanalog){
        adval = analogRead(analogInPin);
      } else {
        adval = SimAnalog();a
      }
      if(rawidx < RAWSAMPLEMAX){
        rawsamples[rawidx] = adval;
        rawidx++;
      }
      if(adval > rawmax) rawmax = adval;
      if(adval < rawmin) rawmin = adval;
      ovsum += adval;  // for mean(32) calculation
      medarray[j] = (float)adval;  // for median(32) calculation
    } // end of oversampling

    x1 = (ovsum / OS);  // This does the mean(32) calculation
    qsort(medarray, OS, sizeof(medarray[0]), sort_descD);
    x2 = medarray[OS / 2];  // This is the median(32) calculation
    mnsamples[i] = x1;
    mdsamples[i] = x2;

    ctimesum += micros() - startctime;
  }// end of sampling for loop


  MeanEtc(mnsamples, SAMPLES, &datAvg1,&std1,&smin1, &smax1);
  pkpk1 = smax1-smin1;
  MeanEtc(mdsamples, SAMPLES, &datAvg2,&std2,&smin2, &smax2);
  pkpk2 = smax2-smin2;
//  Serial.printf("\n Avg. Conversion time: %6.2f uSec\n", ctimesum / SAMPLES);
//  Serial.printf(" Raw min %u   Raw max %u\n", rawmin, rawmax);
  
  if (excelformat) { // print tab separated for direct paste to Excel
    Serial.printf("%8.2f \t%8.2f \t", datAvg1, datAvg2);
    Serial.printf("% 8.2f \t% 8.2f \t", pkpk1, pkpk2);
    Serial.printf("%6.3f\t %6.3f\t ", std1, std2);
    if(useanalog) Serial.print("A"); else Serial.print("S");
    Serial.println();
  } else {  //verbose format on two lines with labels

    Serial.printf(" Avg: %8.2f ", datAvg1);
    Serial.printf(" P-P :%8.2f", pkpk1);
    Serial.printf(" St.Dev: %7.3f    Mean(%u) ", std1, OS);
        if(useanalog) Serial.println("A"); else Serial.println("S");

    Serial.printf(" Avg: %8.2f ", datAvg2);
    Serial.printf(" P-P :%8.2f", pkpk2);
    Serial.printf(" St.Dev: %7.3f    Median(%u)", std2, OS);
    Serial.printf("  ctime: %5.1f\n\n", (float)ctimesum/SAMPLES);
  }

  if(histoflag){
    binMedianF(mnsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin1, (uint16_t)smax1);
    Serial.printf("Mean(%u) method ", OS);
    ShowHisto(histobins, (uint16_t)smin1, (uint16_t)smax1);
    
    binMedianF(mdsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin2, (uint16_t)smax2);
    Serial.printf("Median(%u) method ", OS);
    ShowHisto(histobins, (uint16_t)smin2, (uint16_t)smax2);

    rawmed = binMedianU(rawsamples, RAWSAMPLEMAX, histobins, MAXBINS, (uint16_t)rawmin, (uint16_t)rawmax);
    Serial.printf("Median of Raw values: %u   ",rawmed);
    ShowHisto(histobins, (uint16_t)rawmin, (uint16_t)rawmax);

    histoflag = false;
    Serial.println("\nPress any key to continue.");
    while(!Serial.available());
    ch = Serial.read();
  }
  CheckUserInput();
  
} // end loop()  =====================================================

#define SIMMEAN  20000.0
uint16_t SimAnalog(void){
bool badresult;
uint16_t ival;
float x;
      do {
      badresult = false;
      x = fastrnorm(SIMMEAN, simStd);
      ival = floor(x+0.5);  // round the result
      if (ival > 65535) badresult = true;
      if (ival > (SIMMEAN + 6 * simStd)) badresult = true;
      if (ival < (SIMMEAN - 6 *  simStd)) badresult = true;
    } while (badresult);  // fall through when result in bounds
    return ival;
}


// do the  easy stats mean, median, mode, max, min
// converted input array to float because oversampled mean has fractional part
void MeanEtc(float dat[], uint32_t nvals, float *mnptr, float *stptr, float *minp, float *maxp) {
  uint32_t i;
  float_t imin, imax, ival;
  float  var, fv, s1, s2, mean;
  double  sum, sumsq; // with 4 million samples, even 32-bit floats have rounding problems!
  // NOTE SUMSQ gets LARGE: suppose the values are ~4000 and you sum 100,000 of them then the sumsq
  //       is about (4x10^3) * (4x10^3)  * (1x10^5)   or 1.6x10^12 which is way too big for a uint32_t
  sum = 0.0;
  sumsq = 0.0;
  imin = 65535;
  imax = 0;
  // first get sum and calculate mean
  for (i = 0; i < nvals; i++) {
    ival = dat[i];
    if (ival < imin) imin = ival;
    if (ival > imax) imax = ival;
    sum += (float)ival;
    sumsq += (float)ival * (float)ival;
  }
  mean = sum / nvals;
  // now rcalculate variance in tw-pass algorithm to minimize roundoff error
  s1 = 0; s2 = 0;
  for (i = 0; i < nvals; i++) {
    fv = dat[i] - mean;
    s1 += (fv * fv );
    s2 += fv;
  }
  var = (s1 - s2 * s2 / nvals) / (nvals - 1);
  *mnptr = mean;
  //Serial.printf("Variance: %8.3ef  s1: %8.3e  s2^2/n: %8.3e\n", var, s1, s2*s2/nvals);
  *stptr = sqrt(var);
  *minp = imin;
  *maxp = imax;
}


// Accumulate bin counts for histogram  data should be in range 0..65535
// takes about 140mSec to find median of 100,000 records
// this version for uint16_t dat
uint16_t binMedianU(uint16_t dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
  uint32_t i, bsum, maxbinval;
  uint16_t val;

  if((imax-imin) > numbins){
    Serial.printf("Data range too high for %u bins\n", MAXBINS);
    return 0;
  }
  if(imax<=imin){
    Serial.printf("Max <= Min\n");
    return 0;
  }

  Serial.printf("min: %u   and max %u \n",imin, imax);
  // empty the bins--they could be on the stack and have non-zero values
  memset(bins, 0, numbins * sizeof(uint32_t));
  //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
  //   delay(5);

  //maxbinval = 0;
  for (i = 0; i < numsamp; i++) {
    val = dat[i]; 
   
    if ((val-imin) < numbins) { // prevent writing past end of bins array
      bins[val-imin]++;
    } else printf("Out of Bounds with %u   and min %u\n", val, imin);

  } // bins are filled
  i = 0; bsum = 0;
  do {
    bsum += bins[i];
    i++;
  }  while (bsum < numsamp / 2);

  return imin+ i - 1;
}


// Accumulate bin counts for histogram  data should be in range 0..65535
// takes about 140mSec to find median of 100,000 records
// this version for floating point dat
uint16_t binMedianF(float dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
  uint32_t i, bsum, maxbinval;
  uint16_t val;

  if((imax-imin) > numbins){
    Serial.printf("Data range too high for %u bins\n", MAXBINS);
    return 0;
  }
  if(imax<=imin){
    Serial.printf("Max <= Min\n");
    return 0;
  }

  Serial.printf("min: %u   and max %u \n",imin, imax);
  // empty the bins--they could be on the stack and have non-zero values
  memset(bins, 0, numbins * sizeof(uint32_t));
  //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
  //   delay(5);

  //maxbinval = 0;
  for (i = 0; i < numsamp; i++) {
    val = (uint16_t)dat[i]; // truncate data to integer
//    if(val == imin) Serial.printf( "min of %u at %u\n", val, i);
 //   if(val == imax-1) Serial.printf( "max of %u at %u\n", val, i);
  
    if ((val-imin) < numbins) { // prevent writing past end of bins array
      bins[val-imin]++;
    } else printf("Out of Bounds with %u   and min %u\n", val, imin);

  } // bins are filled
  i = 0; bsum = 0;
  do {
    bsum += bins[i];
    i++;
  }  while (bsum < numsamp / 2);

  return  imin+i - 1;
}


// with new binMedian algorithm, lowest sample value is in bin[bins]
void ShowHisto(uint32_t bins[], uint16_t imin, uint16_t imax) {
  uint16_t i, lnum;
  lnum = 0;
  Serial.print("Histogram Data");
  for (i = 0; i < (imax-imin+1); i++) {
    if ((lnum % 10) == 0) Serial.printf("\n % 4d: ", i+imin);
    lnum++;
    Serial.printf(" % 5lu ", bins[i]);
  }
  Serial.println();
}


// returns a random floating point value between 0.0 and 1.0
// we use the built-in rand() function for speed as we are
// not particularly picky about the characteristics of the number
float randf(void) {
  float val;
  val = (float)rand() / ((float)(RAND_MAX) - 0.5);
  return val;
}
// convert floating rnd to normal curve
float fastrnorm(float mean, float stdDev)
{
  static float spare;
  static float u1;
  static float u2;
  static float s;
  static bool isSpareReady = false;

  if (isSpareReady)
  {
    isSpareReady = false;
    return ((spare * stdDev) + mean);
  } else {
    do {
      u1 = (randf() * 2) - 1;
      u2 = (randf() * 2) - 1;
      s = (u1 * u1) + (u2 * u2);
    } while (s >= 1.0);
    s = sqrtf(-2.0 * logf(s) / s);
    spare = u2 * s;
    isSpareReady = true;
    return (mean + (stdDev * u1 * s));
  }
}
 
So we conclude what I said in #2 - try median and mean to see what works better for you?


Here is more flexible trimmed mean code. I recommend it - it allows one to adjust from mean to median and everything in between. Typically no dither needed. Odd number of samples is better.


Code:
int compare_u16(const void *cmp1, const void *cmp2)
{
  // Need to cast the void *
  const uint16_t a = *((uint16_t *)cmp1);
  const uint16_t b = *((uint16_t *)cmp2);
  return a > b ? -1 : (a < b ? 1 : 0);
}

// take mean of samples on each side of the median - adjustable,  no dither

double trimmed_mean(const uint16_t *array, const unsigned count, const unsigned n)
{
  qsort(array, count, sizeof(array[0]), compare_u16);

  unsigned sum = 0;
  unsigned n2 = 0;

#if 1
  // classic method is to trim n of the most extreme samples from each end - try n = 10% of count
  for (unsigned i = n; i < (count-n); ++i) {
    sum += array[i];
    ++n2;
  }
#else
  // weight each side method - try n = 1 for a tight histogram
  const int median = array[count / 2];
  
  for (unsigned i = 0; i < count; ++i) {
    if (((int)array[i] >= median - (int)n) && ((int)array[i] <= median + (int)n)) {
      sum += array[i];
      ++n2;
    }
  } // for
#endif

  return (double)sum / n2;
}
 
Last edited:
I just notice that the code I posted last night had " analogReadAveraging(4);", which would account for the slower data collection times.

I'm also seeing about 49523 counts for my 2.5V source, compared to 49544 yesterday afternoon. My office is about 10 degrees F. cooler than yesterday afternoon, so perhaps the T3.6 ADC has a significant temperature coefficient or the calibration changes @jonr mentioned are showing up. My LTC6655 voltage reference is supposed to have only about 2PPM/Deg.C temperature coefficient and this change is way beyond that.
 
The reference for the ADC is the VDD (3.3 V) supply -- from a TLV75733P. This only has 1.5 % accuracy, about 200 uV p-p noise in the BW you are using, and probably a change of a few mV over the temperature swing. This will contribute an error in the ADC reading proportional to the input voltage (e.g. at low voltages the error will be small; at near FS, it will be large).
 
Status
Not open for further replies.
Back
Top