Fast Data Logging over USB

Status
Not open for further replies.

Bumbler

Member
I've recently started playing with the Teensy 3.6 and Arduino IDE configured as a 'USB Serial' device at 180MHz, optimized 'Faster'. I am trying to put together a Time of Flight acoustic measuring system which needs quick and well timed analog measurements. So, naturally I've been using Pedvide's ADC library with PDB. So far so good, just measuring the analog write pin to test. Teensy seems to have no trouble keeping up with many hundreds of KSPS. My problem comes from trying to log the data. I'm trying to send the data over USB to my laptop so I can start to run the analysis. I have set up the ADC library to 10 bit measurements, and unless I throttle the sampling frequency to 10 Ksps I cannot get all the data to my computer. I figure that with 2 bytes for data and 1 byte for delimiting per sample, I should be able to transfer at least 300 Ksps, but I drop a lot of data across the USB unless the speed is quite slow. Below is the Teensy Code (many thanks to all the excellent forum posts about ADC):

Code:
#include <ADC.h>
#include <RingBuffer.h>

// connect out_pin to adc_pin, PWM output on out_pin will be measured
// via adc_pin.

const uint8_t adc_pin = A9;
const uint8_t out_pin = 2;

ADC adc;


RingBuffer *buffer = new RingBuffer;

volatile size_t write_pos = 0;
volatile size_t read_pos = 0;

bool flag = false;
bool quitFlag = false;

volatile uint32_t Counter = 0;
uint32_t readCounter = 0;
volatile uint16_t adc_val = 0;

const uint32_t pdb_trigger_frequency = 300000;
const uint32_t seconds = 100;

uint32_t numSamples = pdb_trigger_frequency * seconds;

void setup() {

    
    
    pinMode(adc_pin, INPUT);
    Serial.begin(9600);
    delay(3000);
    Serial.println("Starting");

    adc.setAveraging(1);
    adc.setResolution(10);

    adc.setConversionSpeed(ADC_CONVERSION_SPEED::HIGH_SPEED);
    adc.setSamplingSpeed(ADC_SAMPLING_SPEED::HIGH_SPEED);
    adc.adc0->analogRead(adc_pin); // performs various ADC setup stuff

    analogWriteFrequency(out_pin, 5000);
    analogWrite(out_pin, 128);
    

    delay(2000);
    Serial.println(millis());

    adc.enableInterrupts(ADC_0);
    //NVIC_SET_PRIORITY(IRQ_ADC0,0);
    //NVIC_SET_PRIORITY(IRQ_PDB,2);
    adc.adc0->resetError();
    adc.adc0->stopPDB();
    adc.adc0->startPDB(pdb_trigger_frequency);
    

    

}

void loop() {


      if(readCounter < Counter){
        adc_val = buffer->read();
        Serial.println(adc_val);
        readCounter++;
      }
  else{
    if (quitFlag == true){
      Serial.println("OVER");
      Serial.end(); //Does nothing
      quitFlag = false;
    }
  }
     
}
//void yield(){}

void adc0_isr() {
  if(Counter<numSamples){

    buffer->write(adc.adc0->readSingle());
 
   
  }
      else{
         adc.adc0->stopPDB();
         adc.disableInterrupts(ADC_0);
         adc.adc0->readSingle();
         quitFlag = true;
      }
       Counter++;
}

void pdb_isr(void) {
    PDB0_SC &=~PDB_SC_PDBIF; // clear interrupt
}


For completeness, here is the Python Code (3.7) I am using to catch the data, note that I also attempted this with some other serial monitors (TyCommander among them) and this has been consistently the best for me:


Code:
import time

flag = 1
with open('logFile', 'w') as log:
    time.sleep(1)
    # Don't fight the Kernal, wait some seconds for prepare device

    with open("/dev/ttyACM0", "r") as readBuff:
        while flag:
            ans = readBuff.readline()
            if ans:
                if ans[:-1] == "OVER":
                        flag = 0
                else:
                    log.write(ans[:])
        readBuff.close()
    log.close()



I have also increased the number of NUM_USB_BUFFERS in usb_desc.h to 16 and the number of TX_PACKET_LIMIT in usb_serial.c to 32.

I am wondering if there is something I can do to get at least 250 Ksps over USB?

I guess that the USB transfers are colliding with the ADC ISRs. I have tried to run down other possible issues. I have attempted this on both windows and linux, used the USB data transfer tools Paul provided on PJRC website (just north of 1 MB/s as expected), Serial.write instead of serial.println, different numbers of usb buffers and TX packet limits, changed the interrupt priority of ISRs and PDB, I can certainly make things worse, but I can't seem to get better. What is getting me is that the data loss is generally 1-3% across a range of speeds. The data loss gets worse as the sampling speed increases which makes sense, but it doesn't get much worse.

Here are some results of number of samples transmitted vs received:


Rx.............Tx...........%
---- ............----.........----
10000........10000......1
29900........30000......0.997
49500........50000......0.99
99000........100000....0.99
246000......250000....0.984
295000......300000....0.983
302000......310000....0.974
323500......330000....0.980
340000......350000....0.971

These are rounded and general averages of a number of runs. Mostly I was running at 300 Ksps, and these percentages are the same whether I sample for 1 second or 100 seconds. The losses are random throughout the run.

If anyone has any feedback I would appreciate it. Thanks in advance.
 
Not sure how the RingBuffer works?

The _isr() may be injecting more than one per loop - and loop() is only pulling one each time?

This cannot be correct - but TyCommander prints more data without "OVER" printing until the end.
I picked 21 as the limit on ii because that is close to the ideal USB buffer size of 64 bytes - with 3 bytes per sample, but I oversized the AdcBuff array to allow for data loss as more come in during the write. Then a newline is sent - of course the data is not going out 2 byte binary but as dec to ascii so the byte counts are way off.
Code:
uint16_t AdcBuff[42];
volatile uint32_t ii=0;
void loop() {

        if ( ii > 21 ) {
          for ( int jj=0; jj<11; jj++) {
           Serial.print(AdcBuff[jj]);
           Serial.print(',');
          }
          Serial.println();
          ii=0;
        }

      if(readCounter < Counter){
        // adc_val = buffer->read();
        //AdcBuff[ii++] = buffer->read();
        readCounter++;
      }
  else{
    if (quitFlag == true){
      Serial.println("OVER");
      //Serial.end(); //Does nothing
      quitFlag = false;
    }
  }
     
}
//void yield(){}

void adc0_isr() {
  if(Counter<numSamples){

    // buffer->write(adc.adc0->readSingle());
 AdcBuff[ii++] = adc.adc0->readSingle();
   
  }
      else{
         adc.adc0->stopPDB();
         adc.disableInterrupts(ADC_0);
         adc.adc0->readSingle();
         quitFlag = true;
      }
       Counter++;
}

So ignore that - it was just a test to see data was being lost and watch the code with no idea what it is doing. This should be closer to what you want including the void yield(){}.
The change is adding the "while(readCounter < Counter)". I used comma's because it groups the data better on longer lines in TyCommander with less screen scrolling that will slow it down and make it look jumpy.
It looks fine to the end with a single value printed after the "OVER".
Code:
void loop() {


      if(readCounter < Counter){
      [B]while(readCounter < Counter){[/B]
        adc_val = buffer->read();
        Serial.print(adc_val);
        [B]Serial.print(',');[/B]
        readCounter++;
      }
      Serial.println();
    }
  else{
    if (quitFlag == true){
      Serial.println("OVER");
      Serial.end(); //Does nothing
      quitFlag = false;
    }
  }
     
}
void yield(){}

<edit>: The 'single value printed' after "OVER" was probably an _isr() race where the last data came in after "if(readCounter < Counter)" so the "OVER" test should probably read::
if(readCounter < Counter && quitFlag == true)
 
Last edited:
I modified a program for a test and i am able to send 628571 bytes/sec over USB without any data loss. It only uses 17.8% cpu on a Teensy3.2 at 72Mhz. It's a quite big program i used and it has some overhead, it could probably be faster if it was optimized for the purpose.
I guess Python can be a problem, i made the test with a C++ program on my PC.

This is Linux code but you can do something similar in python or Windows to locate to problem. It will tell you if there are data loss on the PC side.

Code:
void Serial::printUartStats()
{
	struct serial_icounter_struct icount;

	if (ioctl(serial_fd, TIOCGICOUNT, &icount) != -1) {
		printf("frame %u\n", icount.frame );
		printf("parity %u\n", icount.parity );
		printf("overrun %u\n", icount.overrun );
		printf("brk %u\n", icount.brk );
		printf("buf_overrun %u\n", icount.buf_overrun );
		printf("tx %u\n", icount.tx );
		printf("rx %u\n", icount.rx );
	}
}
 
Last edited:
@Jacob - did you try the change from post above? Was that relevant? When I ran the OP code I saw bursts of data between 'OVER' which suggested the data was coming out - but long after the sampling was done.
Code:
void loop() {
    if(readCounter < Counter){
      while(readCounter < Counter){
 
I done some fast testing and you cannot blame the USB. The Ringbuffer is certainly not efficient the default size is only 8 bytes (in my version) witch is useless.
I can recommend a look at https://etlcpp.com. Most of it is good and are difficult to make better.

RingBuffer.h
#define RING_BUFFER_DEFAULT_BUFFER_SIZE 8

The example works much better if i set the ring buffer size to 128 bytes. If i DON'T send anything via USB it will still crash at 300.000 ksps and it will complete at 200.000 ksps. If i do the Serial.println the limit is 150.000 ksps. (compiled with -O3)
I don't knows the specs for the adc, but it look like some issue at 300k samples with the current example.
It's possible to save some bandwidth if you pack the data (6 bits is unused for every sample) and frame then in bigger blocks instead of samples.
My previous test did not depend on the ADC library.
 
Last edited:
You might try this: github.com/tonton81/Circular_Buffer … circular_buffer_example.ino

Still not sure if you tried the while? Given the interrupt there will be incoming samples during the output.

Increased ringbuffer size to 128 and made this change to get a fixed copy of Counter into local var in loop() - seems to run with TyComm:
Code:
void loop() {

  uint32_t ii = Counter;
  if (readCounter < ii) {
    while (readCounter < ii) {
      adc_val = buffer->read();
      Serial.print(adc_val);
      Serial.print(',');
      readCounter++;
    }
    Serial.println();
  }
  else {
    if (readCounter == Counter && quitFlag == true) {
      Serial.println("OVER");
      //Serial.end(); //Does nothing
      quitFlag = false;
    }
  }

}
void yield() {}

Final "if (readCounter == Counter && quitFlag == true) {" noted in post #2 was wrong cut and paste - corrected here.
 
Thanks for the replies. I did forget to state that I did indeed up the ringbuffer size, it was sitting at a ridiculous 32768, so I dropped it back down to 128 (need to remember what I changed when playing).

defragster, to your original post, I expected that at some point my data sampling would overrun my ring buffer, but I'm not seeing the results I would expect if that were the cause of my data drop-outs.

I have tried the while() loop instead of what I posted before and never got better results, before I actually found it to lose more data. When I tried the code you posted however, I got about 295K out of the 300K, so right where I am getting with my own code. Yours does have a benefit of showing where "bunching" of samples is occurring (before each new line) which in the few times I ran it seem to be at random points. At my ridiculous ring buffer size It's a bit interesting that the longest strings of data that I get with your code are hundreds of samples long. One I counted was 850 samples. There was a string of these multiple hundreds of samples in a row before the serial.print() caught back up to the ISR. I'm guessing I was choking up the memory allocations a bit.

At the more manageable 128 ringbuffer size I actually got 297K of the 300K, and the longest string of bunched samples was 30. It does take 6-12 loops before the counter catches back up so something is causing a bit of a backup, may not matter though. I will need to play with it more tomorrow.

Jacob.Schultz, I will need to spend some more time looking at everything you posted. I'm still new to Linux so getting around the ports is taking me longer than I would like.
 
Remember to set your interrupt priorities on the ADC it's important. It makes your first example work without problems. It should be lower than the usb.

NVIC_SET_PRIORITY(IRQ_ADC0, 30);
 
I'm just viewing on Windows with TyComm - but the last posted code looks more consistent in common group sizes - where the while keys on ii captured before the while is entered. USB buffers will send when full so keeping a full one on hand - then breaking often enough to make sure allowing them to send.

Wondering what the ringbuffer does - when underflowed? If it overflows data is lost - but yet in the end as many samples are pulled as pushed in.

It would only take a couple of lines to move to tonton81's CircularBuffer code that may be a bit better behaved ( I hope ) and maybe even faster.
 
Yeah, definitely need to try the new circular buffer code, and maybe rewrite my logging in C++. I made sure my ADC priority was set to 30 as Jacob.Schultz said, but I'm still stuck at 295K Rx for 300K Tx, but if I try on TyCommander, I only get about 200K, so that's a bit weird as well. It'll probably be a couple days before I can run down if it's my software, my hardware, or something else. I'll come back and post when I can find the time to run this down.
 
Given a read by Python code - not human terminal - Transmitting the data as 2 byte Binary instead of ascii could generally cut down on the transfer volume depending on the mix of values sent - especially if sent in groups with fewer terminator characters.
 
Given a read by Python code - not human terminal - Transmitting the data as 2 byte Binary instead of ascii could generally cut down on the transfer volume depending on the mix of values sent - especially if sent in groups with fewer terminator characters.
Though you may need to watch out for certain characters that may/may not be interpreted by the OS (control-s, control-q, newline, tab, etc.). In theory, you should be able to use a RAW protocol and not have to worry about such things, but sometimes they can come back and bite you.
 
Indeed - may need some care and attention for the receiver for both bytes to clear if read as chars not binary (i.e. print() versus write() from Arduino). And still preserve 0xff or something for the sync markers between, but 10 bit analog read values leave room for adjustment.
 
The ringbuffer speed is not bad at all, you can't get it much faster. I compared these two and the speed is identical on ARM. On Intel the ADC circular buffer is 10% faster witch surprised me.

Code:
/*
 * queue.h
 *
 *  Created on: Nov 3, 2018
 *      Author: jsa
 */

#ifndef QUEUE_H_
#define QUEUE_H_

#include <defs.h>

template<class T, int max_size>
class Queue {
public:
	Queue() :
		m_nItems(0),
		m_head(0),
		m_tail(0)
	{}
	void push(const T& item) {
		ASSERT_ID(ERR_QUEUE_OVERFLOW, m_nItems < max_size);
		m_buf[m_tail] = item;
		if (++m_tail == max_size) {
			m_tail = 0;
		}
		m_nItems++;
	}
	void pop() {
		ASSERT_ID(ERR_QUEUE_UNDERFLOW, empty() != 1);
		if (++m_head == max_size) {
			m_head = 0;
		}
		m_nItems--;
	}
	T& front() {
		return m_buf[m_head];
	}
	T& back() {
		return m_buf[ (m_head == 0) ? max_size : m_head - 1];
	}
	bool empty() const {
		return m_nItems == 0;
	}
	bool full() const {
		return m_nItems == max_size;
	}
	uint8_t capacity() const {
		return max_size;
	}
	uint8_t size() const {
		return m_nItems;
	}
	void clear() {
		m_nItems = 0;
		m_head = 0;
		m_tail = 0;
	}
private:
	T m_buf[max_size];
	uint8_t m_nItems;
	uint8_t m_head;
	uint8_t m_tail;
};



#endif /* QUEUE_H_ */
 
Well, I've played a bit more with my code. Python was definitely part of the problem. I switched to the linux shell command: cat /dev/ttyACM0 > logFile.txt
immediately I got about 295500/300000 samples! but then I changed something...
So, now I get about 298-299K/300K samples. Changing back to python dropped me again to 295K/300K, so whatever my Python is doing is wrong.

I have tried before doing serial writes, but they weren't getting me anywhere with the Python and it was more difficult to keep track of the bytes. I'm trying again with C++ now, but saying my C++ is rusty would be generous. So I have a script that will talk to my computer, but I have to work on parsing it and writing to a file, nothing too difficult. I will update when I complete that part.

I did some more playing with the clocking of my teensy, and once I tried overclocking it my samples tanked. I even found that I was getting more stable results with my clock at 144 or 168 MHz rather than 180 MHz. At 120MHz it dropped a bit as well. Does that make sense?
 
I've played with this a little more, still can't pin down my problem. My latest attempt was using Serial.write as opposed to Serial.println and using C++ to capture the data instead of python or cat.


Code:
#include <ADC.h>
#include <RingBuffer.h>
#include <array>

// connect out_pin to adc_pin, PWM output on out_pin will be measured
// via adc_pin.

const uint8_t adc_pin = A9;
const uint8_t out_pin = 2;

ADC adc;
RingBuffer *buffer = new RingBuffer;


uint8_t TxBuffer[64];
uint8_t buffNum = 0;
uint8_t idx = 0;
uint8_t buffRead = 0;
uint8_t endCount = 0;

const uint8_t stopByte[64] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 9, 9, 9, 9};

bool readyRead = false;

bool endFlag = false;
bool quitFlag = false;

uint32_t Counter = 0;
uint32_t readCounter = 0;
uint16_t adc_val = 0;

const uint32_t pdb_trigger_frequency = 100000;
const uint32_t seconds = 1;

uint32_t numSamples = pdb_trigger_frequency * seconds;

void setup() {

    pinMode(adc_pin, INPUT);
    Serial.begin(9600);
    delay(3000);
    //Serial.println("Starting");
    Serial.flush();
    adc.setAveraging(1);
    adc.setResolution(10);

    adc.setConversionSpeed(ADC_CONVERSION_SPEED::HIGH_SPEED);
    adc.setSamplingSpeed(ADC_SAMPLING_SPEED::HIGH_SPEED);
    adc.adc0->analogRead(adc_pin); // performs various ADC setup stuff


    analogWriteFrequency(out_pin, 5000);
    analogWrite(out_pin, 128);

    delay(2000);
    //Serial.println(millis());

    adc.enableInterrupts(ADC_0);
    NVIC_SET_PRIORITY(IRQ_ADC0,30);
    adc.adc0->resetError();
    adc.adc0->stopPDB();
    adc.adc0->startPDB(pdb_trigger_frequency);
    
}
void loop() {





  while(readCounter < Counter){
    
      if(buffNum < 63){
          adc_val = buffer->read();
          TxBuffer[buffNum] = (0x00ff&adc_val);
          buffNum++;
          TxBuffer[buffNum] = (adc_val>>8);
          buffNum++;
          readCounter++;
      }

      
      else{
        Serial.write(TxBuffer,64);
      //  Serial.println();
        buffNum = 0;
      }

  }

    if (readCounter == Counter && quitFlag == true){
      Serial.flush();
      Serial.write(stopByte,64);
      Serial.end(); //Does nothing
      quitFlag = false;
    }
     
}

void yield(){}

void adc0_isr() {
  if(Counter<numSamples){

      buffer->write(adc.adc0->readSingle());
  }
    

      else{
          adc.adc0->stopPDB();
          adc.disableInterrupts(ADC_0);
          adc.adc0->readSingle();
          quitFlag = true;
        
      }
       Counter++;
}

void pdb_isr(void) {
    PDB0_SC &=~PDB_SC_PDBIF; // clear interrupt
}

My C++ code is:

Code:
#include <stdio.h>      
#include <unistd.h>     
#include <fcntl.h>      
#include <errno.h>      
#include <termios.h>    
#include <iostream>
#include <fstream>
#include <string.h>
#include <chrono>
#include <linux/serial.h>
#include <sys/ioctl.h>

using namespace std;


 int main()
{

    
    int  serial_port = open( "/dev/ttyACM0", O_RDWR| O_NOCTTY );
    fcntl(serial_port, F_SETFL, 0);
    
    FILE * pFile;
    pFile = fopen ("logCpp.txt","w");

    //Define the POSIX structure
    struct termios _serial_options;
    struct serial_icounter_struct icount;

    //Read the attribute structure
    tcgetattr(serial_port, &_serial_options);

    //Set the baud rate of the port  to 9600
    cfsetispeed(&_serial_options, B4000000);
    cfsetospeed(&_serial_options, B4000000);
            _serial_options.c_cflag |= (CLOCAL | CREAD);

    //Define other parameters in order to  realize the 8N1 standard
    _serial_options.c_cflag &= ~PARENB;
    _serial_options.c_cflag &= ~CSTOPB;
    _serial_options.c_cflag &= ~CSIZE;
    _serial_options.c_cflag |= CS8;

    //Apply the new attributes 
    tcsetattr(serial_port, TCSANOW, &_serial_options);
    
    errno = 0;
    char read_buffer[64];
   // char subBuff[4] = {0,0,0,0};
  //  char subBuff = 0;
  //  int subIdx = 0;
    bool Flag = 1;
    
    
    auto start = std::chrono::high_resolution_clock::now();
    while(Flag==1){
    int rd = read(serial_port , read_buffer, 64);
    
   /* 
        if (ioctl(serial_port, TIOCGICOUNT, &icount) != -1) {
        printf("frame %u\n", icount.frame );
        printf("parity %u\n", icount.parity );
        printf("overrun %u\n", icount.overrun );
        printf("brk %u\n", icount.brk );
        printf("buf_overrun %u\n", icount.buf_overrun );
        printf("tx %u\n", icount.tx );
        printf("rx %u\n", icount.rx );
    }
    */

    // Display read data
    if (rd > 0)
    {
       // std::cout << rd << " bytes read: ";
        fprintf(pFile," %d", rd);
        fprintf(pFile," %s", " bytes read: ");
        for (int i = 0; i < rd; i++)
        {
            //if(read_buffer[i] ==13){
               // subIdx = 0;
                //int tmp = std::stoi(subBuff);
               // printf(" %d", tmp);
              //  printf(" %d", read_buffer[i]);
             //   std::cout << " ";
            fprintf(pFile,  "%d ", read_buffer[i]);
           // fprintf(pFile," %s", " ");
            
            //}else if(read_buffer[i]!=10){
               // subBuff[subIdx] = read_buffer[i];
               // subIdx++;
            //}
        }
        //fprintf(pFile," %s", " # End ");
        fprintf(pFile," %s \n", " # End");
       // std::cout << " # END" << std::endl;
    }
    // Nothing was to be read
    else if (rd == 0)
    {
        std::cout << "No bytes read =(" << std::endl;
    }
    // Couldn't acces the file for read
    else
    {
        if (errno == EAGAIN || errno == EWOULDBLOCK)
        {
            usleep(20); // Sleep for 20 microseconds.
        }
        else
        {
            printf("%d %s \n", errno, strerror(errno));
        }
    }
    auto stop = std::chrono::high_resolution_clock::now();
    auto duration = std::chrono::duration_cast<std::chrono::seconds>(stop - start); 
    if (duration.count() > 10){
        Flag = 0;
    }
    }
std::cout << std::endl;

}

Anyway, I tried checking the ioctl as Jacob suggested and never saw any overruns.

One thing I did realize which I should have caught on to earlier is that Serial.println actually prints out the values as character arrays. With the analog write shifting between 0 and 1024, I was actually printing out '0' and '1' '0' '2' '4', which leads to an average of 2.5 bytes per reading over the course of the file, not 2 like I had been presuming. So, if anything it's remarkable that i can get close to 300K samples using println. It still doesn't explain why I cannot get anything above 10K samples to read without data loss unfortunately.

I am having trouble with the Serial.write() still. I'm guessing part of it is on my c++ side. If I try and run the Teensy code above and the C++, I get 10-20% of the samples. If I have the arduino serial monitor open while I run the teensy and the c++, I can get above 50%. Still not doing great with it though.

So, I will probably try a fresh install of my teensy libraries to see if I mucked something up along the way.
 
If you are doing 300k samples/sec you are on the limit of available bandwidth. You also have a CR and LF character for every sample. With 5 bytes pr sample (only 3 digits) you are on the limit of a full speed USB. It's more reasonable to use binary transfer.

I don't understand you problem with data loss, i cannot replicate it. I have tried you program several time at 300k samples without any problems.

I can recommend you use cfmakeraw() in you terminal (tty) setup or you will have problems with binary data. Try to read man pages or google. execute "man cfmakeraw". if you have them installed.

Code:
   struct termios tty;

    tcgetattr(serial_fd, &tty) < 0) ;

    cfsetospeed(&tty, baudrate);
    cfsetispeed(&tty, baudrate);

    tty.c_cflag |= (CLOCAL | CREAD);    // ignore modem controls
    tty.c_cflag &= ~CSTOPB;     		// 1 stop bit
    tty.c_cflag &= ~CRTSCTS;    		// no hardware flowcontrol

    cfmakeraw(&tty);	// default is no parity, 8-bit characters

    tcsetattr(serial_fd, TCSANOW, &tty) != 0);
 
Jacob.Schultz,

I have a quick question for you, when you try my code, does your data make sense? I haven't gotten back to my c++ code yet, but I reinstalled arduino and teensyduino to make sure I didn't screw something major up before. I was playing around with the processor speed and while I realize the 3.2 and 3.6 are different, when I slowed down the 3.6 my data stopped making sense. I was still getting about 99% of my data, but the frequency of the measured data was all wrong. With the analog write at 5kHz, I expect about 30 readings of 1023 followed by 30 readings of 0. At lower Mhz, I would get about 11 of 1024 followed by 15 of 0 until I got to about 120k samples. Then it would suddenly shift to 3-4 readings of 1024 and 5-8 of 0 for the remainder. If I tried sampling at 200ksps It would stall at about 150k samples, and at 100ksps It would work as expected (with the 1% data loss). The whole thing would still take 1 sec to run though. So I was just wondering if your 3.2 was measuring appropriately.

Sample datalog of the stall at 120k samples.

Code:
1
1
1023
1023
1023
1023
1023
1023
1023
1023
1023
1023
1023
1
1
1
1
1
1
1
1
1
1
1
1
1
1023
1023
1023
1023
1023
1023
1023
1023
1023
1023
1023
1
1
1
1
1
2
1
1
2
1
1
1
1
1
1
2
1
1023
1023
1023
1
1
1
2
1
1023
1023
1023
1
1
1
2
1
1023
1023
1023
1
1
1
2
1
1023
1023
1023
1
1
 
I found a Teensy 3.6 in my drawer and made a test. The result is the same as before i got exactly 30.000.004 lines every time in the output file, no data loss. I had floating input under the test and the result is between 50-300 most of the time as expected. And no, my output is not changing at different sample rates.

Starting
5300
217
296
192
156
213
279
295
.
.
.
.
.
69
73
54
151
160
123
163
180
198
OVER

I compiled a exact copy of the code below with following options: "Optimize: fastest" and a 180 Mhz cpu. If you don't get the same result there must be something wrong with you setup on the PC side or a side effect of a hardware problem on you Teensy.

Code:
#include <ADC.h>
#include <RingBuffer.h>

// connect out_pin to adc_pin, PWM output on out_pin will be measured
// via adc_pin.

const uint8_t adc_pin = A9;
const uint8_t out_pin = 2;

ADC adc;


RingBuffer *buffer = new RingBuffer;

volatile size_t write_pos = 0;
volatile size_t read_pos = 0;

bool flag = false;
bool quitFlag = false;

volatile uint32_t Counter = 0;
uint32_t readCounter = 0;
volatile uint16_t adc_val = 0;

const uint32_t pdb_trigger_frequency = 300000;
const uint32_t seconds = 100;

uint32_t numSamples = pdb_trigger_frequency * seconds;

void setup() {

    
    
    pinMode(adc_pin, INPUT);
    Serial.begin(9600);
    delay(3000);
    Serial.println("Starting");

    adc.setAveraging(1);
    adc.setResolution(10);

    adc.setConversionSpeed(ADC_CONVERSION_SPEED::HIGH_SPEED);
    adc.setSamplingSpeed(ADC_SAMPLING_SPEED::HIGH_SPEED);
    adc.adc0->analogRead(adc_pin); // performs various ADC setup stuff

    analogWriteFrequency(out_pin, 5000);
    analogWrite(out_pin, 128);
    

    delay(2000);
    Serial.println(millis());

    adc.enableInterrupts(ADC_0);
    NVIC_SET_PRIORITY(IRQ_ADC0,32);
    //NVIC_SET_PRIORITY(IRQ_PDB,2);
    adc.adc0->resetError();
    adc.adc0->stopPDB();
    adc.adc0->startPDB(pdb_trigger_frequency);
    

    

}

void loop() {


      if(readCounter < Counter){
        adc_val = buffer->read();
        Serial.println(adc_val);
        readCounter++;
      }
  else{
    if (quitFlag == true){
      Serial.println("OVER");
      Serial.end(); //Does nothing
      quitFlag = false;
    }
  }
     
}
//void yield(){}

void adc0_isr() {
  if(Counter<numSamples){

    buffer->write(adc.adc0->readSingle());
 
   
  }
      else{
         adc.adc0->stopPDB();
         adc.disableInterrupts(ADC_0);
         adc.adc0->readSingle();
         quitFlag = true;
      }
       Counter++;
}

void pdb_isr(void) {
    PDB0_SC &=~PDB_SC_PDBIF; // clear interrupt
}
 
cfmakeraw did not fix the issue so I ordered a new 3.6 and a 3.2 to test out. Failed on both the new ones.

I switched from my laptop to an older win7 desktop that I built a few years ago. I am able to reliably get 300K readings! Except that I still have two issues, probably related:
It is taking my new code about 7 seconds to transfer the 300K readings (my fault I'm sure, I plan on working on it, won't bother posting the code since it isn't working and not teensy related). And I am still getting the sudden stalled frequency readings. At first I thought that the adc interrupt might be interfering with the analog write, but since I now have plenty of teensy's lying around I used one to just analog write and the other to read and print and got the same behavior. So now I assume that the readings are just too slow, the circular buffer is being overwritten until the 300K samples are reached and the final pattern of the readings are the result. If I can speed up my reading I should be able to validate (since I can't create a buffer large enough to store all readings).

If anyone has any recommendations on a laptop that won't randomly drop usb data I would appreciate it...Not that the past few weeks haven't been fun.
 
So based upon the reading of this thread and my own testing, I too get about 300,000 samples per second. However, the question remains does this mean that if we want to sample across multiple channels we are limited to this upper range? Stated differently, do we need then spread this 300,000 samples per seconds across the number of channels we are interested in sampling?

Logging to the SD card is an option but the file sizes are mondo.

https://forum.pjrc.com/threads/54163-Teensy-3-6-1MHz-sampling-amp-DMA-to-extract-data?highlight=sdfs
 
If you stick to one ADC then absolutely it will be limited across all pins. You can lower the resolution to get more samples per second I think the continuous read sample that comes with teensyduino has the speed tests. You should definitely read up on the whole work up of Pedvide's library:

http://https://forum.pjrc.com/threads/25532-ADC-library-update-now-with-support-for-Teensy-3-1?highlight=pevide+adc

My issue was that the transmission was losing data over USB and that seems to be a motherboard issue with my laptop. Anyway, I seem to remember another forum post that had some discussions of using the PDB with both ADC's. If memory serves, then the PDB didn't play well with both ADC's running and the timings needed extra special care to stay precise. Some more forum reading will certainly help you out.
 
Status
Not open for further replies.
Back
Top