Forum Rule: Always post complete source code & details to reproduce any issue!
Page 2 of 3 FirstFirst 1 2 3 LastLast
Results 26 to 50 of 52

Thread: Checking ADC noise

  1. #26
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    Here is some real, non-shifted data I just took. Intuitively (which may be invalid), I'd expect the correct value to be a little less than 1304. Which is what the median with dither produces.

    Who knows, there could be other algorithms that do better - like a mean after outliers have been removed.

    Histogram:

    1290 0
    1291 1
    1292 0
    1293 1
    1294 1
    1295 0
    1296 1
    1297 0
    1298 2
    1299 0
    1300 0
    1301 0
    1302 5
    1303 125
    1304 370
    1305 50
    1306 8
    1307 0
    1308 2
    1309 0
    1310 1
    1311 0
    1312 0
    1313 0
    1314 1
    1315 0
    1316 0
    1317 1
    1318 1
    1319 0

    Mean: 1304.06
    Median: 1304
    Median with dither: 1303.87
    Mean from 1302 to 1306: 1303.75

  2. #27
    Senior Member
    Join Date
    Jul 2020
    Posts
    406
    The mean is 1303.874 from that data - check your calculations.

  3. #28
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    I didn't include the full histogram, but in doing another test, indeed, the mean was much closer to the dithered median.

    Does anyone have a conclusion - median is better than mean for oversampling or it's just misleading analysis?
    Last edited by jonr; 08-21-2020 at 06:32 PM.

  4. #29
    Thanks Jnor!

    For a measure of DC value, I would definitely choose Median.

    You might get a more realistic value by averaging the centre 3, 4, 5, bins or so.

    But excluding those single count bins that a deviate so much looks like a good idea to me. Those counts orininate form some form of interference IMHO and therefore do not add information about the DC voltage at the ADC input.

  5. #30
    Senior Member
    Join Date
    Jul 2020
    Posts
    406
    My first worry would be the LSB being highly non-random in the outliers, they are nearly all even... Something's up here,
    perhaps a sparkle-code issue?
    https://www.analog.com/media/en/trai...als/MT-011.pdf

  6. #31
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    219
    I updated the test code I posted yesterday based on jonr's post. This version is set up to use the 16-bit setting on the T3.6 ADC, so no dithering is involved. Once again, I used a NIMH cell with 1K in series. I tested with and without 0.1uF from ADC pin to ground---which seemed to make little difference.

    The sampling scheme is the same: Oversample 32 times and compute a sample value from either the median of 32, or the mean of 32.

    2000 of the oversampled values are collected and some statistics displayed. I also added the capability to do a dummy adc conversion before the 32 oversamples. With my signal source it made no difference

    Sample output:
    Code:
     Avg: 26036.15  P-P :   55.84 St.Dev:   6.368    Mean(32) 
     Avg: 26034.83  P-P :   68.00 St.Dev:   9.664    Median(32)
    
     Avg: 26036.67  P-P :   53.31 St.Dev:   6.493    Mean(32) 
     Avg: 26035.34  P-P :   63.00 St.Dev:  10.034    Median(32)
    
     Avg: 26036.63  P-P :   54.31 St.Dev:   6.605    Mean(32) 
     Avg: 26035.41  P-P :   65.00 St.Dev:  10.091    Median(32)
    
     Avg: 26035.59  P-P :   51.94 St.Dev:   5.919    Mean(32) 
     Avg: 26034.47  P-P :   64.00 St.Dev:   9.120    Median(32)
    
     Avg: 26036.62  P-P :   48.25 St.Dev:   6.479    Mean(32) 
     Avg: 26035.23  P-P :   63.00 St.Dev:   9.897    Median(32)
    
     Avg: 26036.26  P-P :   50.94 St.Dev:   6.266    Mean(32) 
     Avg: 26035.00  P-P :   71.00 St.Dev:   9.554    Median(32)
    
     Avg: 26035.11  P-P :   53.06 St.Dev:   5.959    Mean(32) 
     Avg: 26034.19  P-P :   63.00 St.Dev:   8.695    Median(32)
    
     Avg: 26035.28  P-P :   43.22 St.Dev:   5.998    Mean(32) 
     Avg: 26034.20  P-P :   58.00 St.Dev:   8.923    Median(32)
    
     Avg: 26035.20  P-P :   47.28 St.Dev:   6.226    Mean(32) 
     Avg: 26034.26  P-P :   63.00 St.Dev:   8.977    Median(32)
    
     Avg: 26035.64  P-P :   53.44 St.Dev:   6.148    Mean(32) 
     Avg: 26034.69  P-P :   63.00 St.Dev:   9.219    Median(32)
    
     Avg: 26036.09  P-P :   52.66 St.Dev:   5.976    Mean(32) 
     Avg: 26034.94  P-P :   59.00 St.Dev:   8.826    Median(32)
    
     Avg: 26034.96  P-P :   45.53 St.Dev:   5.962    Mean(32) 
     Avg: 26034.00  P-P :   60.00 St.Dev:   8.693    Median(32)
    
     Avg: 26035.83  P-P :   53.25 St.Dev:   5.909    Mean(32) D
     Avg: 26034.70  P-P :   66.00 St.Dev:   8.921    Median(32)
    
     Avg: 26036.94  P-P :   45.84 St.Dev:   6.350    Mean(32) D
     Avg: 26035.59  P-P :   64.00 St.Dev:   9.595    Median(32)
    
     Avg: 26036.48  P-P :   52.81 St.Dev:   6.315    Mean(32) D
     Avg: 26035.32  P-P :   68.00 St.Dev:   9.433    Median(32)
    
     Avg: 26037.01  P-P :   53.81 St.Dev:   6.329    Mean(32) D
     Avg: 26035.81  P-P :   68.00 St.Dev:   9.602    Median(32)
    the D at the end of the line indicates when a dummy conversion was used.

    I saved 32000 of the raw data samples before the oversampling so I could look at the raw data histogram.

    Code:
     Avg: 26034.32  P-P :   48.09 St.Dev:   5.258    Mean(32) D
     Avg: 26033.47  P-P :   57.00 St.Dev:   7.360    Median(32)
    
    Mean(32) method Histogram Data
      26012:      1      0      0      1      2      0      0      7      6      7 
      26022:      6     17     27     35     49     55     69    101    103    137 
      26032:    174    154    168    153    137    151    106     81     50     57 
      26042:     57     27     19     11      9      8      2      5      1      2 
      26052:      1      1      0      1      0      0      1      0      1 
    Median(32) method Histogram Data
      26007:      3      0      0      0      1      1      2      0      6      6 
      26017:      6     14     15     20     20     35     32     62     56     64 
      26027:     83     90     72     76    105    120    118    118     98    114 
      26037:    110     90     60     62     68     59     47     33     27     26 
      26047:     19     17     11     11      5      5      5      2      2      1 
      26057:      0      0      1      0      1      0      0      1 
    First 32000 Raw values Histogram Data
      25858:      0      0      0      0      0      0      0      0      0      0 
      25868:      0      0      0      0      0      0      0      0      0      0 
      25878:      0      0      0      0      0      0      0      0      0      0 
      25888:      0      0      0      0      1      0      0      0      0      0 
      25898:      0      0      0      0      0      0      0      0      0      0 
      25908:      0      0      0      0      0      2      0      0      0      0 
      25918:      0      1      0      0      1      0      0      1      0      2 
      25928:      0      0      1      1      0      0      1      1      1      3 
      25938:      0      2      0      2      1      1      1      1      1      2 
      25948:      4      2      4      3      4      7      3      5      4      2 
      25958:      6      7     14     11     17     26     15      9     13     35 
      25968:     33     31     30     48     42     44     49     57     65     58 
      25978:     67     92     83     84     81    114    129    161    119    178 
      25988:    155    155    181    168    219    210    217    258     90    158 
      25998:    183    236    337    267    276    317    299    332    342    296 
      26008:    342    343    294    190    240    203    240    244    357    372 
      26018:    379    354    416    358    356    345    443    373    364    355 
      26028:    426    338    319    351    415    373    368    360    392    344 
      26038:    356    304    396    387    333    476    322    295    324    307 
      26048:    424    355    315    372    380    345    363    310    439    338 
      26058:    373    361    390    334    308    289    365    327    317    332 
      26068:    301    292    275    238    255    271    226    201    135    161 
      26078:    167    166    230    199    167    154    169    133    115    104 
      26088:    132     76     68     65     78     46     39     29     40     30 
      26098:     15     23     17     10     13     10     10      6      5      6 
      26108:      4      2      3      3      1      3      1      1      2      2 
      26118:      2      2      2      3      0      1      3      0      0      1 
      26128:      0      1      1      1      1      0      2      1      0      0 
      26138:      1      2      1      0      2      0      0      0      0      0 
      26148:      0      1      0      1      0      0      1      0      0      0 
      26158:      0      0      0      0      0      0      0      0      0      0 
      26168:      0      0      0      0      0      0      0      0      0      0 
      26178:      0      0      0      0      0      0      0      1
    One thing that stood out is that the sample averages from the mean(32) method are higher than those from the median(32) by about 1 count. I've yet to decide whether that's a problem in my calculations or an intrinsic difference based on the sample distribution. In this version of the program both the mean(32) and median(32) statistics are derived from the same oversampled data in one pass.

    Collecting and saving one oversampled value takes about 200microseconds, so this collection will run out of steam at about 4K Samples/second on at T3.6 at 180MHz.

    Here's the source code:
    Code:
    // Analog input test for Teensy 4.X   Oct 4 2012 J.Beale
    // Hacked on by JonR (probably the source of any problems)
    // Setup: https://picasaweb.google.com/109928236040342205185/Electronics#5795546092126071650
    // Posted to PJRC form 8/20/20
    // Modified by mborgerson 8/20/20
    // Changed for T3.6 and 16-bit input 8/21/2020
    // Switched to collecting samples and after-the-fact statistics
    // Added histogram display of samples and raw data
    
    #define VREF (3.2444)         // ADC reference voltage (= power supply)
    #define VINPUT (1.04020)      // ADC input voltage from resistive divider to VREF
    #define ADCMAX (65536)        // maximum possible reading from ADC
    #define EXPECTED (ADCMAX*(VINPUT/VREF))    // expected ADC reading
    
    #define OS     32            // how many times to oversample 
    #define OFFSET  0 // -149.76  // calibrate out a fixed offset
    #define num 1                 // ADC internal oversampling
    
    const int analogInPin = A8;   // Analog input pin
    const int ledpin = 13;
    
    #define MAXBINS 1000
    uint32_t histobins[MAXBINS];
    
    #define SAMPLES 2000         // how many samples to combine for pp, std.dev statistics
    #define RAWSAMPLEMAX 32000
    float mnsamples[SAMPLES], mdsamples[SAMPLES];
    float rawsamples[RAWSAMPLEMAX];
    uint16_t rawmin,rawmax;
    #define SAMPLERATE  1000
    bool histoflag = false;
    bool excelformat = false;
    bool dummyconvert = false;
    
    elapsedMicros sampletimer;
    #define  LEDON digitalWriteFast(ledpin, HIGH);
    #define  LEDOFF digitalWriteFast(ledpin, LOW);
    
    void setup() {    // ==============================================================
      pinMode(analogInPin, INPUT_DISABLE); // because I don't know default at startup
      pinMode(ledpin, OUTPUT);
      analogReadAveraging(num);
      analogReadRes(16);  // Teensy 3.6: set ADC resolution to this many bits
      // on T4.X, Default Arduino AnalogRead(pin) takes about 6uSec---the slowest mode!
      Serial.begin(115200);      // baud rate is ignored with Teensy USB ACM i/o
      delay(1000);
      Serial.printf("# Teensy 3.6 16-bit ADC test start: %dx%d oversampling \n\n", OS, num);
      delay(1000);
      srand(micros()); // use unix time as random seed
      Serial.println();
    
    } // ==== end setup() ===========
    
    
    // for T3.6 with 16-bit ADC, samples are float
    int sort_descD(const void *cmp1, const void *cmp2)
    {
      // Need to cast the void * to uint16_t
      float a = *((float *)cmp1);
      float b = *((float *)cmp2);
      // The comparison
      return a > b ? -1 : (a < b ? 1 : 0);
    }
    
    
    // Added fixed timing for oversampled  acquisition
    void loop() {  // ================================================================
    
      float datAvg1, datAvg2, pkpk1, pkpk2,std1,std2;
      float smin1,smax1, smin2,smax2;
      float ctimesum;
      uint32_t startctime;
      uint16_t adval;
      char ch;
      float x1,x2, ovsum;
      uint16_t rawidx;
    
      float array[OS];  // used to qsort 32 oversampled adc values
      ctimesum = 0; // for accumulating sum of conversion and calculation times
      sampletimer = 0;  // reset the sampling timer
      rawidx = 0;  rawmin = 65535;  rawmax = 0;
      for (int i = 0; i < SAMPLES; i++) {
        // wait until time for next sample
        while (sampletimer < (1000000 / SAMPLERATE));
        sampletimer = 0;  // reset the sampling timer
        startctime = micros();
        ovsum = 0;
    
        // do a first conversion to remove first-sample bias on higher impedance source      
        if(dummyconvert) adval = analogRead(analogInPin);
        for (int j = 0; j < OS; ++j) {
          adval = analogRead(analogInPin);
          if(rawidx < RAWSAMPLEMAX){
            rawsamples[rawidx] = adval;
            rawidx++;
          }
          if(adval > rawmax) rawmax = adval;
          if(adval < rawmin) rawmin = adval;
          ovsum += adval;  // for mean(32) calculation
          array[j] = adval;  // for median(32) calculation
        } // end of oversampling
    
        x1 = (ovsum / OS);  // This does the mean(32) calculation
        qsort(array, OS, sizeof(array[0]), sort_descD);
        x2 = array[OS / 2];  // This is the median(32) calculation
        mnsamples[i] = x1;
        mdsamples[i] = x2;
    
        ctimesum += micros() - startctime;
      }// end of sampling for loop
    
    
      MeanEtc(mnsamples, SAMPLES, &datAvg1,&std1,&smin1, &smax1);
      pkpk1 = smax1-smin1;
      MeanEtc(mdsamples, SAMPLES, &datAvg2,&std2,&smin2, &smax2);
      pkpk2 = smax2-smin2;
    //  Serial.printf("\n Avg. Conversion time: %6.2f uSec\n", ctimesum / SAMPLES);
    //  Serial.printf(" Raw min %u   Raw max %u\n", rawmin, rawmax);
      
      if (excelformat) { // print tab separated for direct paste to Excel
        Serial.printf("%8.2f \t%8.2f \t", datAvg1, datAvg2);
        Serial.printf("% 8.2f \t% 8.2f \t", pkpk1, pkpk2);
        Serial.printf("%6.3f\t %6.3f\t ", std1, std2);
        if(dummyconvert) Serial.print("D");
        Serial.println();
      } else {  //verbose format on two lines with labels
    
        Serial.printf(" Avg: %8.2f ", datAvg1);
        Serial.printf(" P-P :%8.2f", pkpk1);
        Serial.printf(" St.Dev: %7.3f    Mean(32) ", std1);
        if(dummyconvert) Serial.println("D"); else Serial.println();
    
        Serial.printf(" Avg: %8.2f ", datAvg2);
        Serial.printf(" P-P :%8.2f", pkpk2);
        Serial.printf(" St.Dev: %7.3f    Median(32)\n\n", std2);
      }
    
      if(histoflag){
        binMedian(mnsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin1, (uint16_t)smax1+1);
        Serial.print("Mean(32) method ");
        ShowHisto(histobins, (uint16_t)smin1, (uint16_t)smax1+1);
        
        binMedian(mdsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin2, (uint16_t)smax2+1);
        Serial.print("Median(32) method ");
        ShowHisto(histobins, (uint16_t)smin2, (uint16_t)smax2+1);
    
        binMedian(rawsamples, RAWSAMPLEMAX, histobins, MAXBINS, (uint16_t)rawmin, (uint16_t)rawmax+1);
        Serial.print("First 32000 Raw values ");
        ShowHisto(histobins, (uint16_t)rawmin, (uint16_t)rawmax+1);
    
        histoflag = false;
        Serial.println("\nPress any key to continue.");
        while(!Serial.available());
        ch = Serial.read();
      }
      CheckUserInput();
      
    } // end loop()  =====================================================
    
    // Simple user interface to switch between mean and median of 32 samples
    // and to turn dithering on and off
    void CheckUserInput(void) {
      char ch;
      if (Serial.available()) {
        ch = Serial.read();
        if (ch == 'e') excelformat = !excelformat;
        if (ch == 'd')  dummyconvert  = !dummyconvert;
        if (ch == 'h') histoflag = true;
      }
    }
    
    
    // do the  easy stats mean, median, mode, max, min
    // converted input array to float because oversampled mean has fractional part
    void MeanEtc(float dat[], uint32_t nvals, float *mnptr, float *stptr, float *minp, float *maxp) {
      uint32_t i;
      float_t imin, imax, ival;
      float  var, fv, s1, s2, mean;
      double  sum, sumsq; // with 4 million samples, even 32-bit floats have rounding problems!
      // NOTE SUMSQ gets LARGE: suppose the values are ~4000 and you sum 100,000 of them then the sumsq
      //       is about (4x10^3) * (4x10^3)  * (1x10^5)   or 1.6x10^12 which is way too big for a uint32_t
      sum = 0.0;
      sumsq = 0.0;
      imin = 65535;
      imax = 0;
      // first get sum and calculate mean
      for (i = 0; i < nvals; i++) {
        ival = dat[i];
        if (ival < imin) imin = ival;
        if (ival > imax) imax = ival;
        sum += (float)ival;
        sumsq += (float)ival * (float)ival;
      }
      mean = sum / nvals;
      // now rcalculate variance in tw-pass algorithm to minimize roundoff error
      s1 = 0; s2 = 0;
      for (i = 0; i < nvals; i++) {
        fv = dat[i] - mean;
        s1 += (fv * fv );
        s2 += fv;
      }
      var = (s1 - s2 * s2 / nvals) / (nvals - 1);
      *mnptr = mean;
      //Serial.printf("Variance: %8.3ef  s1: %8.3e  s2^2/n: %8.3e\n", var, s1, s2*s2/nvals);
      *stptr = sqrt(var);
      *minp = imin;
      *maxp = imax;
    }
    
    
    // Accumulate bin counts for histogram  data should be in range 0..65535
    // takes about 140mSec to find median of 100,000 records
    uint16_t binMedian(float dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
      uint32_t i, bsum, maxbinval;
      uint16_t val;
    
      if((imax-imin) > numbins){
        Serial.printf("Data range too high for %u bins\n", MAXBINS);
        return 0;
      }
      if(imax<=imin){
        Serial.printf("Max <= Min\n");
        return 0;
      }
    
      // empty the bins--they could be on the stack and have non-zero values
      memset(bins, 0, numbins * sizeof(uint32_t));
      //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
      //   delay(5);
    
      //maxbinval = 0;
      for (i = 0; i < numsamp; i++) {
        val = (uint16_t)dat[i]+0.5; // round data to integer
        if ((val-imin) < numbins) { // prevent writing past end of bins array
          bins[val-imin]++;
     //     if(bins[val-imin > maxbinval){
     //       *modeptr = val;
     //       maxbinval = bins[val];
     //    }
        }
    
      } // bins are filled
      i = 0; bsum = 0;
      do {
        bsum += bins[i];
        i++;
      }  while (bsum < numsamp / 2);
    
      return i - 1;
    }
    
    // with new binMedian algorithm, lowest sample value is in bin[0]
    void ShowHisto(uint32_t bins[], uint16_t imin, uint16_t imax) {
      uint16_t i, lnum;
      lnum = 0;
      Serial.print("Histogram Data");
      for (i = 0; i < (imax-imin); i++) {
        if ((lnum % 10) == 0) Serial.printf("\n % 4d: ", i+imin);
        lnum++;
        Serial.printf(" % 5lu ", bins[i]);
      }
      Serial.println();
    }

  7. #32
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    219
    The difference between the mean and median algorithms gets worse when I collect from a voltage divider source consisting of two 50K resistors.
    Code:
     Avg: 32719.75  P-P :   92.62 St.Dev:  13.380    Mean(32) 
     Avg: 32716.36  P-P :  143.00 St.Dev:  21.539    Median(32)
    
    Mean(32) method Histogram Data
      32677:      1      0      2      0      0      2      1      1      1      2 
      32687:      0      5      6      3     11      5      8     12     10     15 
      32697:     13     11     22     19     29     31     29     33     27     47 
      32707:     48     34     53     41     70     45     50     50     58     53 
      32717:     51     55     65     47     74     51     60     52     55     53 
      32727:     50     54     48     44     36     46     38     33     19     24 
      32737:     23     12     21     19     16     13     16     12      8     11 
      32747:     11      3      4      4      6      3      4      3      1      2 
      32757:      0      1      0      1      0      0      0      0      1      0 
      32767:      1      0      0      1 
    Median(32) method Histogram Data
      32635:      1      0      0      0      0      1      0      0      0      0 
      32645:      0      0      0      0      0      0      0      0      0      0 
      32655:      1      1      0      0      1      0      0      0      2      0 
      32665:      1      0      3      0      1      3      3      3      3      2 
      32675:      2      3      8     10      4     13      6     15     14     11 
      32685:     16     21     27     19     20     27     27     28     17     33 
      32695:     18     25     17     31     32     35     37     21     25     26 
      32705:     19     43     36     34     35     40     22     31     26     38 
      32715:     35     29     36     40     34     37     32     50     37     38 
      32725:     45     39     36     27     23     43     33     32     22     10 
      32735:     15     19     19     23     21     19     15     23     11     12 
      32745:     10     14      7     19     12      9      9     11      7     12 
      32755:     11      9      4      4      8      3      2      6      5      4 
      32765:      6      4      6      5      1      9      0      1      4      2 
      32775:      2      0      0      1 
    First 32000 Raw values Histogram Data
      32459:      0      0      0      0      0      0      0      0      0      0 
      32469:      0      0      0      0      0      0      0      0      0      0 
      32479:      0      0      0      0      0      0      0      0      0      0 
      32489:      0      0      0      0      0      0      0      0      0      0 
      32499:      0      0      1      0      0      0      0      1      1      0 
      32509:      1      1      1      1      0      1      0      1      2      0 
      32519:      0      0      0      1      1      3      1      2      0      1 
      32529:      3      2      2      2      2      1      3      2      2      0 
      32539:      3      3      8      2      9      4      5      6      9      5 
      32549:      7      4      5     11      5     12     14     14     12     11 
      32559:     18      5      8     19     17      7     20     19     17     15 
      32569:     21     18     19     13     29     25     29     24     29     43 
      32579:     34     28     38     30     42     30     30     55     49     32 
      32589:     34     63     50     45     35     57     59     53     64     64 
      32599:     60     59     47     68     68     46     62     52     59     61 
      32609:     53     92     82     68     66     89     71     76     62    114 
      32619:     83     90     72    145    109     90     84    121    101    101 
      32629:    114    146    119    120     94    159    107    108    156    119 
      32639:    110    126     83    172    121    116    135    139    139    116 
      32649:    121    148    150    133    135    174    155    125    134    158 
      32659:    151    130    120    170    135    132    123    182    153    109 
      32669:    135     85    111    111    112    170    129    132    125    162 
      32679:    144    126    114    185    143    133    121    178    147    112 
      32689:    102    186    125    131    119    147    119    116    103    159 
      32699:    147    122    175    115    100    115    109    170    124    110 
      32709:    106    156    107    120     98    145    134    117    132    148 
      32719:    119    111    113    163    138    141    139    136    122    125 
      32729:    105    148    108    108    101     72     76    113    107    155 
      32739:    125    110    128    155    126    102    109    154    132     94 
      32749:    119    184    120    115    122    161    149    135    121    158 
      32759:    141    131    124    193    159    159    231    137    141    123 
      32769:    129    178    145    147    168    161    145    153    118    172 
      32779:    154    138    129    181    136    132    142    164    144    132 
      32789:    157    153    125    135    109    155    116    105    101     89 
      32799:    117    123    114    147    113    116    104    114    104    130 
      32809:     97    158    116    103    105    147    106     87     89    151 
      32819:    115     96     97    121     82     87     68    109     91     87 
      32829:     91     73     69     73     55     87     77     65     60     65 
      32839:     72     72     41     69     63     52     54     57     43     37 
      32849:     43     48     41     46     39     38     39     33     34     27 
      32859:     37     25     26     13     23     22     17     33     26     18 
      32869:     18     18     19     15      7     16     15     17      8      7 
      32879:     13     14      8     12     15      8     10      8      6      5 
      32889:      6      7      7      5      4      5      7      1      8      3 
      32899:      5      2      3      3      2      2      2      1      3      1 
      32909:      0      3      1      1      0      2      1      0      2      1 
      32919:      2      0      1      2      0      0      0      0      2      0 
      32929:      0      0      0      0      0      1      0      0      0      2 
      32939:      0      0      0      0      0      0      0      0      0

  8. #33
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    That latest software won't run on my teensy 3.1 due to memory, but with my sw version and hardware, median slightly outperforms mean.

  9. #34
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    219
    Quote Originally Posted by jonr View Post
    That latest software won't run on my teensy 3.1 due to memory, but with my sw version and hardware, median slightly outperforms mean.
    OOPS! I copied the code before I changed the data type for rawsamples from float to uint16_t. The following code should work, and I updated some of the histogram routines to make sure the max and min values were displayed.

    Code:
    // Analog input test for Teensy 4.X   Oct 4 2012 J.Beale
    // Hacked on by JonR (probably the source of any problems)
    // Setup: https://picasaweb.google.com/109928236040342205185/Electronics#5795546092126071650
    // Posted to PJRC form 8/20/20
    // Modified by mborgerson 8/20/20
    // Changed for T3.6 and 16-bit input 8/21/2020
    // Switched to collecting samples and after-the-fact statistics
    // Added histogram display of samples and raw data
    
    #define VREF (3.2444)         // ADC reference voltage (= power supply)
    #define VINPUT (1.04020)      // ADC input voltage from resistive divider to VREF
    #define ADCMAX (65536)        // maximum possible reading from ADC
    #define EXPECTED (ADCMAX*(VINPUT/VREF))    // expected ADC reading
    
    #define OS     32            // how many times to oversample 
    #define OFFSET  0 // -149.76  // calibrate out a fixed offset
    #define num 1                 // ADC internal oversampling
    
    const int analogInPin = A8;   // Analog input pin
    const int ledpin = 13;
    
    #define MAXBINS 1000
    uint32_t histobins[MAXBINS];
    
    #define SAMPLES 2000         // how many samples to combine for pp, std.dev statistics
    #define RAWSAMPLEMAX 64000
    float mnsamples[SAMPLES], mdsamples[SAMPLES];
    uint16_t rawsamples[RAWSAMPLEMAX];
    uint16_t rawmin,rawmax;
    #define SAMPLERATE  500
    bool histoflag = false;
    bool excelformat = false;
    bool dummyconvert = false;
    
    elapsedMicros sampletimer;
    #define  LEDON digitalWriteFast(ledpin, HIGH);
    #define  LEDOFF digitalWriteFast(ledpin, LOW);
    
    void setup() {    // ==============================================================
      pinMode(analogInPin, INPUT_DISABLE); // because I don't know default at startup
      pinMode(ledpin, OUTPUT);
      analogReadAveraging(4);
      analogReadRes(16);  // Teensy 3.6: set ADC resolution to this many bits
      // on T4.X, Default Arduino AnalogRead(pin) takes about 6uSec---the slowest mode!
      Serial.begin(115200);      // baud rate is ignored with Teensy USB ACM i/o
      delay(1000);
      Serial.printf("# Teensy 3.6 16-bit ADC test start: %dx%d oversampling \n\n", OS, num);
      delay(1000);
      srand(micros()); // use unix time as random seed
      Serial.println();
    
    } // ==== end setup() ===========
    
    
    // for T3.6 with 16-bit ADC, samples are float
    int sort_descD(const void *cmp1, const void *cmp2)
    {
      // Need to cast the void * to uint16_t
      float a = *((float *)cmp1);
      float b = *((float *)cmp2);
      // The comparison
      return a > b ? -1 : (a < b ? 1 : 0);
    }
    
    
    // Added fixed timing for oversampled  acquisition
    void loop() {  // ================================================================
    
      float datAvg1, datAvg2, pkpk1, pkpk2,std1,std2;
      float smin1,smax1, smin2,smax2;
      float ctimesum;
      uint32_t startctime;
      uint16_t adval;
      char ch;
      float x1,x2, ovsum;
      uint16_t rawidx, rawmed;
    
      float array[OS];  // used to qsort 32 oversampled adc values
      ctimesum = 0; // for accumulating sum of conversion and calculation times
      sampletimer = 0;  // reset the sampling timer
      rawidx = 0;  rawmin = 65535;  rawmax = 0;
      for (int i = 0; i < SAMPLES; i++) {
        // wait until time for next sample
        while (sampletimer < (1000000 / SAMPLERATE));
        sampletimer = 0;  // reset the sampling timer
        startctime = micros();
        ovsum = 0;
    
        // do a first conversion to remove first-sample bias on higher impedance source      
        if(dummyconvert) adval = analogRead(analogInPin);
        for (int j = 0; j < OS; ++j) {
          adval = analogRead(analogInPin);
          if(rawidx < RAWSAMPLEMAX){
            rawsamples[rawidx] = adval;
            rawidx++;
          }
          if(adval > rawmax) rawmax = adval;
          if(adval < rawmin) rawmin = adval;
          ovsum += adval;  // for mean(32) calculation
          array[j] = (float)adval;  // for median(32) calculation
        } // end of oversampling
    
        x1 = (ovsum / OS);  // This does the mean(32) calculation
        qsort(array, OS, sizeof(array[0]), sort_descD);
        x2 = array[OS / 2];  // This is the median(32) calculation
        mnsamples[i] = x1;
        mdsamples[i] = x2;
    
        ctimesum += micros() - startctime;
      }// end of sampling for loop
    
    
      MeanEtc(mnsamples, SAMPLES, &datAvg1,&std1,&smin1, &smax1);
      pkpk1 = smax1-smin1;
      MeanEtc(mdsamples, SAMPLES, &datAvg2,&std2,&smin2, &smax2);
      pkpk2 = smax2-smin2;
    //  Serial.printf("\n Avg. Conversion time: %6.2f uSec\n", ctimesum / SAMPLES);
    //  Serial.printf(" Raw min %u   Raw max %u\n", rawmin, rawmax);
      
      if (excelformat) { // print tab separated for direct paste to Excel
        Serial.printf("%8.2f \t%8.2f \t", datAvg1, datAvg2);
        Serial.printf("% 8.2f \t% 8.2f \t", pkpk1, pkpk2);
        Serial.printf("%6.3f\t %6.3f\t ", std1, std2);
        if(dummyconvert) Serial.print("D");
        Serial.println();
      } else {  //verbose format on two lines with labels
    
        Serial.printf(" Avg: %8.2f ", datAvg1);
        Serial.printf(" P-P :%8.2f", pkpk1);
        Serial.printf(" St.Dev: %7.3f    Mean(32) ", std1);
        if(dummyconvert) Serial.println("D"); else Serial.println();
    
        Serial.printf(" Avg: %8.2f ", datAvg2);
        Serial.printf(" P-P :%8.2f", pkpk2);
        Serial.printf(" St.Dev: %7.3f    Median(32)\n\n", std2);
      }
    
      if(histoflag){
        binMedianF(mnsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin1, (uint16_t)smax1);
        Serial.print("Mean(32) method ");
        ShowHisto(histobins, (uint16_t)smin1, (uint16_t)smax1);
        
        binMedianF(mdsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin2, (uint16_t)smax2);
        Serial.print("Median(32) method ");
        ShowHisto(histobins, (uint16_t)smin2, (uint16_t)smax2);
    
        rawmed = binMedianU(rawsamples, RAWSAMPLEMAX, histobins, MAXBINS, (uint16_t)rawmin, (uint16_t)rawmax);
        Serial.printf("Median of Raw values: %u   ",rawmed);
        ShowHisto(histobins, (uint16_t)rawmin, (uint16_t)rawmax);
    
        histoflag = false;
        Serial.println("\nPress any key to continue.");
        while(!Serial.available());
        ch = Serial.read();
      }
      CheckUserInput();
      
    } // end loop()  =====================================================
    
    // Simple user interface to switch between mean and median of 32 samples
    // and to turn dithering on and off
    void CheckUserInput(void) {
      char ch;
      if (Serial.available()) {
        ch = Serial.read();
        if (ch == 'e') excelformat = !excelformat;
        if (ch == 'd')  dummyconvert  = !dummyconvert;
        if (ch == 'h') histoflag = true;
      }
    }
    
    
    // do the  easy stats mean, median, mode, max, min
    // converted input array to float because oversampled mean has fractional part
    void MeanEtc(float dat[], uint32_t nvals, float *mnptr, float *stptr, float *minp, float *maxp) {
      uint32_t i;
      float_t imin, imax, ival;
      float  var, fv, s1, s2, mean;
      double  sum, sumsq; // with 4 million samples, even 32-bit floats have rounding problems!
      // NOTE SUMSQ gets LARGE: suppose the values are ~4000 and you sum 100,000 of them then the sumsq
      //       is about (4x10^3) * (4x10^3)  * (1x10^5)   or 1.6x10^12 which is way too big for a uint32_t
      sum = 0.0;
      sumsq = 0.0;
      imin = 65535;
      imax = 0;
      // first get sum and calculate mean
      for (i = 0; i < nvals; i++) {
        ival = dat[i];
        if (ival < imin) imin = ival;
        if (ival > imax) imax = ival;
        sum += (float)ival;
        sumsq += (float)ival * (float)ival;
      }
      mean = sum / nvals;
      // now rcalculate variance in tw-pass algorithm to minimize roundoff error
      s1 = 0; s2 = 0;
      for (i = 0; i < nvals; i++) {
        fv = dat[i] - mean;
        s1 += (fv * fv );
        s2 += fv;
      }
      var = (s1 - s2 * s2 / nvals) / (nvals - 1);
      *mnptr = mean;
      //Serial.printf("Variance: %8.3ef  s1: %8.3e  s2^2/n: %8.3e\n", var, s1, s2*s2/nvals);
      *stptr = sqrt(var);
      *minp = imin;
      *maxp = imax;
    }
    
    
    // Accumulate bin counts for histogram  data should be in range 0..65535
    // takes about 140mSec to find median of 100,000 records
    // this version for uint16_t dat
    uint16_t binMedianU(uint16_t dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
      uint32_t i, bsum, maxbinval;
      uint16_t val;
    
      if((imax-imin) > numbins){
        Serial.printf("Data range too high for %u bins\n", MAXBINS);
        return 0;
      }
      if(imax<=imin){
        Serial.printf("Max <= Min\n");
        return 0;
      }
    
      Serial.printf("min: %u   and max %u \n",imin, imax);
      // empty the bins--they could be on the stack and have non-zero values
      memset(bins, 0, numbins * sizeof(uint32_t));
      //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
      //   delay(5);
    
      //maxbinval = 0;
      for (i = 0; i < numsamp; i++) {
        val = dat[i]; 
       
        if ((val-imin) < numbins) { // prevent writing past end of bins array
          bins[val-imin]++;
        } else printf("Out of Bounds with %u   and min %u\n", val, imin);
    
      } // bins are filled
      i = 0; bsum = 0;
      do {
        bsum += bins[i];
        i++;
      }  while (bsum < numsamp / 2);
    
      return imin+ i - 1;
    }
    
    
    // Accumulate bin counts for histogram  data should be in range 0..65535
    // takes about 140mSec to find median of 100,000 records
    // this version for floating point dat
    uint16_t binMedianF(float dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
      uint32_t i, bsum, maxbinval;
      uint16_t val;
    
      if((imax-imin) > numbins){
        Serial.printf("Data range too high for %u bins\n", MAXBINS);
        return 0;
      }
      if(imax<=imin){
        Serial.printf("Max <= Min\n");
        return 0;
      }
    
      Serial.printf("min: %u   and max %u \n",imin, imax);
      // empty the bins--they could be on the stack and have non-zero values
      memset(bins, 0, numbins * sizeof(uint32_t));
      //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
      //   delay(5);
    
      //maxbinval = 0;
      for (i = 0; i < numsamp; i++) {
        val = (uint16_t)dat[i]; // truncate data to integer
    //    if(val == imin) Serial.printf( "min of %u at %u\n", val, i);
     //   if(val == imax-1) Serial.printf( "max of %u at %u\n", val, i);
      
        if ((val-imin) < numbins) { // prevent writing past end of bins array
          bins[val-imin]++;
        } else printf("Out of Bounds with %u   and min %u\n", val, imin);
    
      } // bins are filled
      i = 0; bsum = 0;
      do {
        bsum += bins[i];
        i++;
      }  while (bsum < numsamp / 2);
    
      return  imin+i - 1;
    }
    
    
    // with new binMedian algorithm, lowest sample value is in bin[bins]
    void ShowHisto(uint32_t bins[], uint16_t imin, uint16_t imax) {
      uint16_t i, lnum;
      lnum = 0;
      Serial.print("Histogram Data");
      for (i = 0; i < (imax-imin+1); i++) {
        if ((lnum % 10) == 0) Serial.printf("\n % 4d: ", i+imin);
        lnum++;
        Serial.printf(" % 5lu ", bins[i]);
      }
      Serial.println();
    }

  10. #35
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    Got it to run by reducing RAWSAMPLEMAX. A typical T3.1 result:

    Avg: 21024.89 P-P : 23.81 St.Dev: 3.713 Mean(32)
    Avg: 21025.63 P-P : 11.00 St.Dev: 2.505 Median(32)

  11. #36
    Great work, but I am very curious how the ADC performs with a low impedance source, likr 10R or so. A real low noise stable, low impedance source..... cannt you repeat this with a much larger capacitor to gnd?

  12. #37
    I am sure you can get St.Dev down to 1 with a larger oversampling number, like 128 or so. Then you would have 16bit performace.

  13. #38
    Senior Member
    Join Date
    Jul 2020
    Posts
    406
    Quote Originally Posted by tschrama View Post
    I am sure you can get St.Dev down to 1 with a larger oversampling number, like 128 or so. Then you would have 16bit performace.
    Depends what you mean by 16 bit performance - resolution perhaps, but there's integral non-linearity, and drift to consider, and
    the circuit needs to be good enough too (precision voltage reference, ground loops avoided, etc). And this is DC performance only
    rather than dynamic (much harder to get dynamic performance of course).

  14. #39
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    > Depends what you mean by 16 bit performance ...

    Exactly. For some uses, a teensy can be similar to a good external 16 bit converter if you over sample by 4000x. In other ways, no amount of oversampling gets there.

    In my tests, the T4 outperforms the T3. The latter introduces a big offset every time it re-calibrates. It probably is possible to write a lower noise calibration routine.

  15. #40
    Quote Originally Posted by MarkT View Post
    Depends what you mean by 16 bit performance - resolution perhaps, but there's integral non-linearity, and drift to consider, and
    the circuit needs to be good enough too (precision voltage reference, ground loops avoided, etc). And this is DC performance only
    rather than dynamic (much harder to get dynamic performance of course).
    When I mean 16bit performance, I mean performance equivalent to a typical 16bit ADC IC. Once you get standard deviation of about 2^-15 FullScale, by averaging multiple sample, (e.g. 256 or so) you have the equivalent of a 16bit ADC IC. This has nothing to do with the dynamics of a signal, not with drift, nor with offset.

  16. #41
    Quote Originally Posted by jonr View Post
    > Depends what you mean by 16 bit performance ...

    Exactly. For some uses, a teensy can be similar to a good external 16 bit converter if you over sample by 4000x. In other ways, no amount of oversampling gets there.

    In my tests, the T4 outperforms the T3. The latter introduces a big offset every time it re-calibrates. It probably is possible to write a lower noise calibration routine.
    I see no problem with averaging 4000 samples, allthough 256 should be enough to get the StDev down to 2^-15 FS . I regulairy average traces of 40e3 samples, sampled at 200MHz with a 14bit ADC, 10e3..40e3 times to fish low level signals out of noisy signals.

  17. #42
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    Here is some "trimmed mean" code that often does better than either median or mean.

    Code:
    int compare_u16(const void *cmp1, const void *cmp2)
    {
      // Need to cast the void * 
      const uint16_t a = *((uint16_t *)cmp1);
      const uint16_t b = *((uint16_t *)cmp2);
      return a > b ? -1 : (a < b ? 1 : 0);
    }
    
    // take mean of samples on each side of the median
    double trimmed_mean(const uint16_t *array, const unsigned count, const unsigned n)
    {
      qsort(array, count, sizeof(array[0]), compare_u16);
    
      unsigned sum = 0;
      unsigned n2 = 0;
      const int median = array[count / 2];
    
      const unsigned high = array[count/2 - 1];  // caution: decending sort
      const unsigned low = array[count/2 + 1];
    
      for (unsigned i = 0; i < count; ++i) {
        //Serial.printf("%u ",array[i]);
        if ((array[i] >= median - n) && (array[i] <= median + n)) {  // alternative method
        //if ((array[i] >= low) && (array[i] <= high)) {
          sum += array[i];
          ++n2;
        }
      } // for
    
      //Serial.printf(" med = %u, sum = %u, n = %u\n",median, sum,n2);
    
      return (double)sum / n2;
    }

  18. #43
    Senior Member
    Join Date
    Jul 2020
    Posts
    406
    Quote Originally Posted by tschrama View Post
    When I mean 16bit performance, I mean performance equivalent to a typical 16bit ADC IC. Once you get standard deviation of about 2^-15 FullScale, by averaging multiple sample, (e.g. 256 or so) you have the equivalent of a 16bit ADC IC. This has nothing to do with the dynamics of a signal, not with drift, nor with offset.
    Not sure there's such a thing as a typical 16bit ADC. For instance audio class ADCs have very different performance
    criteria than instrumentation class ADCs, and there are very different implementation methods for precision ADCs,
    each with their own strengths/weaknesses - SAR and sigma/delta and flash techniques are all used as are many
    proprietry hybrid approaches.

    I think you are talking about DC accuracy or perhaps DC repeatability? Dual slope integrating ADCs are actually a pretty
    good choice for this and offer more than 16 bits of true accuracy too. https://www.ti.com/lit/an/snoa597b/snoa597b.pdf

    For varying signals this isn't realistic, so an SAR ADC might be the way to go, and a typical 16 bit SAR might be accurate
    (or at least linear) to 14.5 bits or so with a really good voltage reference and attention to detail. You won't get anything
    like that from the T4 built-in ADC, merely the illusion of accuracy (ie short-term repeatability). Same goes for any 10 bit
    ADC without a good voltage reference. The 10 bit staircase is not going to be accurate or stable to 16 bits, that would
    require a different process for making the die for one thing, with laser trimming and so forth.

  19. #44
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    219
    Quote Originally Posted by jonr View Post
    > Depends what you mean by 16 bit performance ...

    Exactly. For some uses, a teensy can be similar to a good external 16 bit converter if you over sample by 4000x. In other ways, no amount of oversampling gets there.

    In my tests, the T4 outperforms the T3. The latter introduces a big offset every time it re-calibrates. It probably is possible to write a lower noise calibration routine.
    If the errors are in the ADC, not in the input multiplexers, you could get better performance on the T3 by using two spare channels, one for a stable 2.5V reference and another for the same reference divided down to something near your lowest signal of interest. (Hopefully that lowest signal is well above ground so you don't have to worry about clipping.) At regular intervals, you could sample the two reference channels and use them for a two-point calibration curve to correct the signal on your sensor channel(s). If that works out, you can start worrying about the temperature stability of your sensors! ;-)

  20. #45
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    > At regular intervals, you could sample the two reference channels and use them for a two-point calibration ...

    Even before every sample. Your thinking is similar to a lock-in amplifier, which can pull clean signals out of incredible amounts of noise - even when the noise is about the same (or lower) frequency as the signal.

  21. #46
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    Would be interesting to know if the analog comparator on the teensy could be used to implement a dual-slope integrating ADC that outperformed the built-in ADC (at low speeds).

  22. #47
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    219
    I updated my test code with the following 'features':
    1. You can switch between simulated normally-distributed data with the 'a' key.
    2. The simulated data has a variable std deviation, which you can change: '4':4.0, '5':8,'6':16,'7':32.
    3. You can vary the number of oversamples. '1':32, '2':128,'3':1024. Note that you can't maintain 1KHz at the higher oversample rates with the slow ADC settings. The simulated data generator is faster, though.

    I built a stable low-noise voltage source with an LTC6655 2.5V reference chip that has been trying to escape my parts bin since 2014. The proto board connects with short connections to avoid noise pickup. See Photo:

    Click image for larger version. 

Name:	VSource.jpg 
Views:	9 
Size:	122.1 KB 
ID:	21444

    The voltage source connects 2.5V to A9 and 1.0V to A8--the latter so I can test the ADC with the 1.2V internal reference.

    Here are the results of some tests:
    Code:
    Simulated normally-distributed data oversampled 32 times, stats for 2000 samples.
    
     Avg: 19999.99  P-P :   10.28 St.Dev:   1.399    Mean(32) S
     Avg: 19999.70  P-P :   12.00 St.Dev:   1.777    Median(32)  ctime: 114.2
    
     Avg: 19999.94  P-P :    9.12 St.Dev:   1.391    Mean(32) S
     Avg: 19999.64  P-P :   13.00 St.Dev:   1.770    Median(32)  ctime: 114.2
    
    min: 19995   and max 20004 
    Mean(32) method Histogram Data
      19995:      3     36    129    320    514    531    342    105     18      2 
    min: 19993   and max 20006 
    Median(32) method Histogram Data
      19993:      1      2     13     63    155    270    415    462    314    213 
      20003:     70     20      1      1 
    min: 19963   and max 20030 
    Median of Raw values: 20000   Histogram Data
      19963:      2      0      1      0      3      1      1      2      4      8 
      19973:     13     24     27     45     61     75    115    157    193    255 
      19983:    316    416    564    722    887   1063   1237   1453   1631   1941 
      19993:   2132   2382   2643   2879   2990   3071   3184   3233   3110   3100 
      20003:   3017   2793   2620   2489   2170   1958   1680   1412   1237   1008 
      20013:    843    636    531    428    339    249    191    151    106     66 
      20023:     46     26     20     21     11      7      2      2 
    
    Analog data from 2.5V LTC6655 voltage refrence.
    
     Avg: 49553.83  P-P :    6.19 St.Dev:   0.857    Mean(32) A
     Avg: 49559.06  P-P :    8.00 St.Dev:   1.030    Median(32)  ctime: 590.1
    
     Avg: 49553.85  P-P :    5.81 St.Dev:   0.854    Mean(32) A
     Avg: 49559.17  P-P :    6.00 St.Dev:   1.031    Median(32)  ctime: 590.1
    
    min: 49551   and max 49557 
    Mean(32) method Histogram Data
      49551:     29    291    815    678    170     16      1 
    min: 49556   and max 49562 
    Median(32) method Histogram Data
      49556:      2     90    416    747    569    153     23 
    min: 49487   and max 49569 
    Median of Raw values: 49559   Histogram Data
      49487:      1      8     14     49    120    185    271    341    378    263 
      49497:    186    108     48     20      6      2      0      2      7     19 
      49507:     52    129    214    330    387    346    231    158     78     30 
      49517:     12      4      8     23     48     76    144    213    230    267 
      49527:    249    257    185    130     93     58     42     59    101    170 
      49537:    242    285    309    274    245    185    176    206    252    341 
      49547:    371    447    484    496    610    666    793   1045   1448   2204 
      49557:   3625   5614   7447   8577   8076   6301   3858   1870    830    256 
      49567:     83     25      7
    It's apparent that the simulated data has a nice normal distribution and the mean and median results are well within 1LSB of each other. The ADC results are NOT normally distributed and the mean and median method results are far from each other. I measured the 3.3V rail as 3.3065V. When I plug that number and 2.5002 V at the input, the theoretical number of counts the ADC should see is 49,554.8.
    The mean method gives a result within 1LSB at 49553.85. The median method gives a number about 4 counts higher. The mean method shows about 6 counts P-P noise, (about 300 microVolts). The noise specification for the LTC6655 is 0.67PPM at 1KHz and even lower at the oversampling frequency due to the 10uF cap on the output. That low noise figure and the skewed distribution lead me to believe that the noise is in the ADC, not my voltage source.

    Here's the source code so you can try it out with your own sources and sensors:
    Code:
    // Analog input test for Teensy 4.X   Oct 4 2012 J.Beale
    // Hacked on by JonR (probably the source of any problems)
    // Setup: https://picasaweb.google.com/109928236040342205185/Electronics#5795546092126071650
    // Posted to PJRC form 8/20/20
    // Modified by mborgerson 8/20/20
    // Changed for T3.6 and 16-bit input 8/21/2020
    // Switched to collecting samples and after-the-fact statistics
    // Added histogram display of samples and raw data
    
    #define VREF (3.2444)         // ADC reference voltage (= power supply)
    #define VINPUT (1.04020)      // ADC input voltage from resistive divider to VREF
    #define ADCMAX (65536)        // maximum possible reading from ADC
    #define EXPECTED (ADCMAX*(VINPUT/VREF))    // expected ADC reading
    
    
    #define OFFSET  0 // -149.76  // calibrate out a fixed offset
    #define num 1                 // ADC internal oversampling
    
    const int analogInPin = A9;   // Analog input pin
    const int ledpin = 13;
    
    #define MAXBINS 1000
    uint32_t histobins[MAXBINS];
    
    #define OSMAX 1024
    /*  for later implementation of running mean
    uint16_t rmdat[MAXOS];
    uint16_t rmidx = 0;
    uint32_t rmsum;
    *********************************************/
    float simStd = 8.0;
    
    #define SAMPLES 2000         // how many samples to combine for pp, std.dev statistics
    #define RAWSAMPLEMAX 64000
    
    uint16_t OS = 32;
    float mnsamples[SAMPLES], mdsamples[SAMPLES];
    uint16_t rawsamples[RAWSAMPLEMAX];
    uint16_t rawmin,rawmax;
    #define SAMPLERATE  500
    bool histoflag = false;
    bool excelformat = false;
    bool useanalog = false;
    
    elapsedMicros sampletimer;
    #define  LEDON digitalWriteFast(ledpin, HIGH);
    #define  LEDOFF digitalWriteFast(ledpin, LOW);
    
    void setup() {    // ==============================================================
      pinMode(analogInPin, INPUT_DISABLE); // because I don't know default at startup
      pinMode(ledpin, OUTPUT);
      analogReadAveraging(4);
      analogReadRes(16);  // Teensy 3.6: set ADC resolution to this many bits
      // on T4.X, Default Arduino AnalogRead(pin) takes about 6uSec---the slowest mode!
      Serial.begin(115200);      // baud rate is ignored with Teensy USB ACM i/o
      delay(1000);
      Serial.printf("# Teensy 3.6 16-bit ADC test start: %dx%d oversampling \n\n", OS, num);
      delay(1000);
      srand(micros()); // use unix time as random seed
      Serial.println();
      ARM_DEMCR |= ARM_DEMCR_TRCENA;
      ARM_DWT_CTRL |= ARM_DWT_CTRL_CYCCNTENA;
    
    } // ==== end setup() ===========
    
    
    // for T3.6 with 16-bit ADC, samples are float because mean and running mean return values with fractional part
    int sort_descD(const void *cmp1, const void *cmp2)
    {
      // Need to cast the void * to uint16_t
      float a = *((float *)cmp1);
      float b = *((float *)cmp2);
      // The comparison
      return a > b ? -1 : (a < b ? 1 : 0);
    }
    
    // Simple user interface to switch between mean and median of 32 samples
    // and to turn dithering on and off
    void CheckUserInput(void) {
      char ch;
      if (Serial.available()) {
        ch = Serial.read();
        if (ch == 'e') excelformat = !excelformat;
        if (ch == 'a') useanalog = !useanalog;
        if (ch == 'h') histoflag = true;
        if (ch == '1') OS = 32;
        if (ch == '2') OS = 128;
        if (ch == '3') OS = 1024;
        if (ch == '4') simStd = 4.0;
        if (ch == '5') simStd = 8.0;
        if (ch == '6') simStd = 16.0;
        if (ch == '7') simStd = 32.0;   
        srand(micros()); // reseed the RNG
      }
    }
    
     float medarray[OSMAX];  // used to qsort oversampled adc values
    
    //  uint32_t dwt;
     // dwt = ARM_DWT_CYCCNT;
    
    // Added fixed timing for oversampled  acquisition
    void loop() {  // ================================================================
    
      float datAvg1, datAvg2, pkpk1, pkpk2,std1,std2;
      float smin1,smax1, smin2,smax2;
      float ctimesum;
      uint32_t startctime;
      uint16_t adval;
      char ch;
      float x1,x2, ovsum;
      uint16_t rawidx, rawmed;
    
     
      ctimesum = 0; // for accumulating sum of conversion and calculation times
      sampletimer = 0;  // reset the sampling timer
      rawidx = 0;  rawmin = 65535;  rawmax = 0;
      for (int i = 0; i < SAMPLES; i++) {
        // wait until time for next sample
        while (sampletimer < (1000000 / SAMPLERATE));
        sampletimer = 0;  // reset the sampling timer
        startctime = micros();
        ovsum = 0;
    
        for (int j = 0; j < OS; ++j) {
          if(useanalog){
            adval = analogRead(analogInPin);
          } else {
            adval = SimAnalog();a
          }
          if(rawidx < RAWSAMPLEMAX){
            rawsamples[rawidx] = adval;
            rawidx++;
          }
          if(adval > rawmax) rawmax = adval;
          if(adval < rawmin) rawmin = adval;
          ovsum += adval;  // for mean(32) calculation
          medarray[j] = (float)adval;  // for median(32) calculation
        } // end of oversampling
    
        x1 = (ovsum / OS);  // This does the mean(32) calculation
        qsort(medarray, OS, sizeof(medarray[0]), sort_descD);
        x2 = medarray[OS / 2];  // This is the median(32) calculation
        mnsamples[i] = x1;
        mdsamples[i] = x2;
    
        ctimesum += micros() - startctime;
      }// end of sampling for loop
    
    
      MeanEtc(mnsamples, SAMPLES, &datAvg1,&std1,&smin1, &smax1);
      pkpk1 = smax1-smin1;
      MeanEtc(mdsamples, SAMPLES, &datAvg2,&std2,&smin2, &smax2);
      pkpk2 = smax2-smin2;
    //  Serial.printf("\n Avg. Conversion time: %6.2f uSec\n", ctimesum / SAMPLES);
    //  Serial.printf(" Raw min %u   Raw max %u\n", rawmin, rawmax);
      
      if (excelformat) { // print tab separated for direct paste to Excel
        Serial.printf("%8.2f \t%8.2f \t", datAvg1, datAvg2);
        Serial.printf("% 8.2f \t% 8.2f \t", pkpk1, pkpk2);
        Serial.printf("%6.3f\t %6.3f\t ", std1, std2);
        if(useanalog) Serial.print("A"); else Serial.print("S");
        Serial.println();
      } else {  //verbose format on two lines with labels
    
        Serial.printf(" Avg: %8.2f ", datAvg1);
        Serial.printf(" P-P :%8.2f", pkpk1);
        Serial.printf(" St.Dev: %7.3f    Mean(%u) ", std1, OS);
            if(useanalog) Serial.println("A"); else Serial.println("S");
    
        Serial.printf(" Avg: %8.2f ", datAvg2);
        Serial.printf(" P-P :%8.2f", pkpk2);
        Serial.printf(" St.Dev: %7.3f    Median(%u)", std2, OS);
        Serial.printf("  ctime: %5.1f\n\n", (float)ctimesum/SAMPLES);
      }
    
      if(histoflag){
        binMedianF(mnsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin1, (uint16_t)smax1);
        Serial.printf("Mean(%u) method ", OS);
        ShowHisto(histobins, (uint16_t)smin1, (uint16_t)smax1);
        
        binMedianF(mdsamples, SAMPLES, histobins, MAXBINS, (uint16_t)smin2, (uint16_t)smax2);
        Serial.printf("Median(%u) method ", OS);
        ShowHisto(histobins, (uint16_t)smin2, (uint16_t)smax2);
    
        rawmed = binMedianU(rawsamples, RAWSAMPLEMAX, histobins, MAXBINS, (uint16_t)rawmin, (uint16_t)rawmax);
        Serial.printf("Median of Raw values: %u   ",rawmed);
        ShowHisto(histobins, (uint16_t)rawmin, (uint16_t)rawmax);
    
        histoflag = false;
        Serial.println("\nPress any key to continue.");
        while(!Serial.available());
        ch = Serial.read();
      }
      CheckUserInput();
      
    } // end loop()  =====================================================
    
    #define SIMMEAN  20000.0
    uint16_t SimAnalog(void){
    bool badresult;
    uint16_t ival;
    float x;
          do {
          badresult = false;
          x = fastrnorm(SIMMEAN, simStd);
          ival = floor(x+0.5);  // round the result
          if (ival > 65535) badresult = true;
          if (ival > (SIMMEAN + 6 * simStd)) badresult = true;
          if (ival < (SIMMEAN - 6 *  simStd)) badresult = true;
        } while (badresult);  // fall through when result in bounds
        return ival;
    }
    
    
    // do the  easy stats mean, median, mode, max, min
    // converted input array to float because oversampled mean has fractional part
    void MeanEtc(float dat[], uint32_t nvals, float *mnptr, float *stptr, float *minp, float *maxp) {
      uint32_t i;
      float_t imin, imax, ival;
      float  var, fv, s1, s2, mean;
      double  sum, sumsq; // with 4 million samples, even 32-bit floats have rounding problems!
      // NOTE SUMSQ gets LARGE: suppose the values are ~4000 and you sum 100,000 of them then the sumsq
      //       is about (4x10^3) * (4x10^3)  * (1x10^5)   or 1.6x10^12 which is way too big for a uint32_t
      sum = 0.0;
      sumsq = 0.0;
      imin = 65535;
      imax = 0;
      // first get sum and calculate mean
      for (i = 0; i < nvals; i++) {
        ival = dat[i];
        if (ival < imin) imin = ival;
        if (ival > imax) imax = ival;
        sum += (float)ival;
        sumsq += (float)ival * (float)ival;
      }
      mean = sum / nvals;
      // now rcalculate variance in tw-pass algorithm to minimize roundoff error
      s1 = 0; s2 = 0;
      for (i = 0; i < nvals; i++) {
        fv = dat[i] - mean;
        s1 += (fv * fv );
        s2 += fv;
      }
      var = (s1 - s2 * s2 / nvals) / (nvals - 1);
      *mnptr = mean;
      //Serial.printf("Variance: %8.3ef  s1: %8.3e  s2^2/n: %8.3e\n", var, s1, s2*s2/nvals);
      *stptr = sqrt(var);
      *minp = imin;
      *maxp = imax;
    }
    
    
    // Accumulate bin counts for histogram  data should be in range 0..65535
    // takes about 140mSec to find median of 100,000 records
    // this version for uint16_t dat
    uint16_t binMedianU(uint16_t dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
      uint32_t i, bsum, maxbinval;
      uint16_t val;
    
      if((imax-imin) > numbins){
        Serial.printf("Data range too high for %u bins\n", MAXBINS);
        return 0;
      }
      if(imax<=imin){
        Serial.printf("Max <= Min\n");
        return 0;
      }
    
      Serial.printf("min: %u   and max %u \n",imin, imax);
      // empty the bins--they could be on the stack and have non-zero values
      memset(bins, 0, numbins * sizeof(uint32_t));
      //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
      //   delay(5);
    
      //maxbinval = 0;
      for (i = 0; i < numsamp; i++) {
        val = dat[i]; 
       
        if ((val-imin) < numbins) { // prevent writing past end of bins array
          bins[val-imin]++;
        } else printf("Out of Bounds with %u   and min %u\n", val, imin);
    
      } // bins are filled
      i = 0; bsum = 0;
      do {
        bsum += bins[i];
        i++;
      }  while (bsum < numsamp / 2);
    
      return imin+ i - 1;
    }
    
    
    // Accumulate bin counts for histogram  data should be in range 0..65535
    // takes about 140mSec to find median of 100,000 records
    // this version for floating point dat
    uint16_t binMedianF(float dat[], uint32_t numsamp, uint32_t bins[], uint16_t numbins, uint16_t imin, uint16_t imax) {
      uint32_t i, bsum, maxbinval;
      uint16_t val;
    
      if((imax-imin) > numbins){
        Serial.printf("Data range too high for %u bins\n", MAXBINS);
        return 0;
      }
      if(imax<=imin){
        Serial.printf("Max <= Min\n");
        return 0;
      }
    
      Serial.printf("min: %u   and max %u \n",imin, imax);
      // empty the bins--they could be on the stack and have non-zero values
      memset(bins, 0, numbins * sizeof(uint32_t));
      //  Serial.printf("In binMedian, address of dat is % p\n", &dat);
      //   delay(5);
    
      //maxbinval = 0;
      for (i = 0; i < numsamp; i++) {
        val = (uint16_t)dat[i]; // truncate data to integer
    //    if(val == imin) Serial.printf( "min of %u at %u\n", val, i);
     //   if(val == imax-1) Serial.printf( "max of %u at %u\n", val, i);
      
        if ((val-imin) < numbins) { // prevent writing past end of bins array
          bins[val-imin]++;
        } else printf("Out of Bounds with %u   and min %u\n", val, imin);
    
      } // bins are filled
      i = 0; bsum = 0;
      do {
        bsum += bins[i];
        i++;
      }  while (bsum < numsamp / 2);
    
      return  imin+i - 1;
    }
    
    
    // with new binMedian algorithm, lowest sample value is in bin[bins]
    void ShowHisto(uint32_t bins[], uint16_t imin, uint16_t imax) {
      uint16_t i, lnum;
      lnum = 0;
      Serial.print("Histogram Data");
      for (i = 0; i < (imax-imin+1); i++) {
        if ((lnum % 10) == 0) Serial.printf("\n % 4d: ", i+imin);
        lnum++;
        Serial.printf(" % 5lu ", bins[i]);
      }
      Serial.println();
    }
    
    
    // returns a random floating point value between 0.0 and 1.0
    // we use the built-in rand() function for speed as we are
    // not particularly picky about the characteristics of the number
    float randf(void) {
      float val;
      val = (float)rand() / ((float)(RAND_MAX) - 0.5);
      return val;
    }
    // convert floating rnd to normal curve
    float fastrnorm(float mean, float stdDev)
    {
      static float spare;
      static float u1;
      static float u2;
      static float s;
      static bool isSpareReady = false;
    
      if (isSpareReady)
      {
        isSpareReady = false;
        return ((spare * stdDev) + mean);
      } else {
        do {
          u1 = (randf() * 2) - 1;
          u2 = (randf() * 2) - 1;
          s = (u1 * u1) + (u2 * u2);
        } while (s >= 1.0);
        s = sqrtf(-2.0 * logf(s) / s);
        spare = u2 * s;
        isSpareReady = true;
        return (mean + (stdDev * u1 * s));
      }
    }

  23. #48
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    650
    So we conclude what I said in #2 - try median and mean to see what works better for you?


    Here is more flexible trimmed mean code. I recommend it - it allows one to adjust from mean to median and everything in between. Typically no dither needed. Odd number of samples is better.


    Code:
    int compare_u16(const void *cmp1, const void *cmp2)
    {
      // Need to cast the void *
      const uint16_t a = *((uint16_t *)cmp1);
      const uint16_t b = *((uint16_t *)cmp2);
      return a > b ? -1 : (a < b ? 1 : 0);
    }
    
    // take mean of samples on each side of the median - adjustable,  no dither
    
    double trimmed_mean(const uint16_t *array, const unsigned count, const unsigned n)
    {
      qsort(array, count, sizeof(array[0]), compare_u16);
    
      unsigned sum = 0;
      unsigned n2 = 0;
    
    #if 1
      // classic method is to trim n of the most extreme samples from each end - try n = 10% of count
      for (unsigned i = n; i < (count-n); ++i) {
        sum += array[i];
        ++n2;
      }
    #else
      // weight each side method - try n = 1 for a tight histogram
      const int median = array[count / 2];
      
      for (unsigned i = 0; i < count; ++i) {
        if (((int)array[i] >= median - (int)n) && ((int)array[i] <= median + (int)n)) {
          sum += array[i];
          ++n2;
        }
      } // for
    #endif
    
      return (double)sum / n2;
    }
    Last edited by jonr; 08-23-2020 at 02:10 PM.

  24. #49
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    219
    I just notice that the code I posted last night had " analogReadAveraging(4);", which would account for the slower data collection times.

    I'm also seeing about 49523 counts for my 2.5V source, compared to 49544 yesterday afternoon. My office is about 10 degrees F. cooler than yesterday afternoon, so perhaps the T3.6 ADC has a significant temperature coefficient or the calibration changes @jonr mentioned are showing up. My LTC6655 voltage reference is supposed to have only about 2PPM/Deg.C temperature coefficient and this change is way beyond that.

  25. #50
    Senior Member Jp3141's Avatar
    Join Date
    Nov 2012
    Posts
    486
    The reference for the ADC is the VDD (3.3 V) supply -- from a TLV75733P. This only has 1.5 % accuracy, about 200 uV p-p noise in the BW you are using, and probably a change of a few mV over the temperature swing. This will contribute an error in the ADC reading proportional to the input voltage (e.g. at low voltages the error will be small; at near FS, it will be large).

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •