Serial1.available - counting bytes in the FIFO

[defragster]

Question: Is there a way to force Serial1 to on command empty the FIFO into the buffered data? .available() only returns bytes currently buffered. I am assuming this is behind the issue I see below.

Issue: I'm trying to get a timestamp to the start of data arriving ( from a GPS ). Given the baud rate on the port - and given there are 10 bit cycles per character at default N,8,1 - if I know how many bytes have been completed I can guess the start time using simple math of the bytes showing available referred to a running elapsedMicros timer tsGPS:

#define GPS_BAUD   460800
      GPS_Avail = Serial1.available();
      GPS_ = tsGPS - ((1000000*( (GPS_Avail*10) ) ) / GPS_BAUD ); // converted to microseconds from current elapsedMicros tsGPS

This is very accurate based on alternate solution info below as reference. But .available() only counts the bytes already pulled from the FIFO, so some messages are off by 1 to 4 whole characters.

I have a more accurate solution working well on T_3.6 at 180 or 240 MHz - but it is getting throttled and occasionally missing ( <1% failure based from other activity ) on the T_3.5 (or T_3.6) running at 120 MHz suggesting it is on the edge as written - or the interrupt is getting skipped even when I upped port priority. I started the CPU CycleCounter as a fast way to ignore intermediate RISING interrupts after the 5 Hz 100 byte message begins and it works reliably. So on the faster T_3.6 I am running both solutions side by side and that is where I see the bytes not accounted for in .availalble() calc above. The average error is 13 uS, but runs from 0 to an extreme of 84 uS - which would be 4 characters.

// setup() //   attachInterrupt(digitalPinToInterrupt( 9 ), GPS_serialrx_isr, RISING);

FASTRUN void GPS_serialrx_isr() {
  static uint32_t tt;
  static uint32_t xx = 0;
  tt = ARM_DWT_CYCCNT;
  if ( tt - xx > RxGapTime ) {
    GPS_RX_time = tsGPS; // Start bit transition detected after 'idle' time // GPS_RX_Time = PartSec();
    GPS_newData=1;
  }
  xx = tt;
}

If I can get the first above solution working to include FIFO bytes it would involve way less system overhead as this serialrx_isr() fires multiple times during the 100 byte message, when I only need the first each 200 ms.

FAILED HACK: I thought calling the uart0_status_isr() would blindly empty all FIFO bytes, but that doesn't seem to be doing the trick as I expected - perhaps because it is under HighWater mark?

Thus the question: Is there a way to count the FIFO bytes - or to trigger the FIFO to empty into the buffered data to be counted? Or is there something else I am missing?

 

[Chris O.]

I think this is what you're looking for:
UART_RCFIFO*-*UART*FIFO*Receive*Count
I think in Teensy serial1 (void uart0_status_isr(void)) implementation it looks like:
avail = UART0_RCFIFO; //UART 0 FIFO Receive Count

 

[defragster]

I'll have to give it another look - my first edit didn't change to add UART0_RCFIFO to the count didn't change anything. But I can look at the FM there.

I saw that in the uart0_status_isr() and hoped it would pick it up - but since the FIFO is under the HighWater it skips that - which if I read right makes general sense as pulling those bytes might overflow RAM buffer.

Also in the PJRC source for that is a note of a critical TripWire - pinging that stuff when empty can set an IDLE int flag so I thought it best to ask before trying something doomed to fail.

At some point those bytes need to roll over from the FIFO - I suppose that is when the UART goes idle?

 

[Chris O.]

At some point those bytes need to roll over from the FIFO - I suppose that is when the UART goes idle?

Not only if you look at this it means:

If (UART0_S1 & (UART_S1_RDRF | UART_S1_IDLE)) { //UART Status Register 1-(RDRF)Receive Data Register Full Flag or (IDLE) Idle Line Flag
__disable_irq();
avail = UART0_RCFIFO;

Basically it means if we get Idle or fifo water mark reached, trigger the interrupt (uart0_status_isr)

 

[defragster]

@Chris 0. :: You gave the right solution! This is shared code where I debug out Serial1 and GPS on Serial2 - With Serial1 on the brain ( and #defines hiding values) I was wrongly applying this FIX on UART0==Serial1 {as posted in my query ).

Thanks, when properly applied to UART1_RCFIFO as : GPS_State = tsGPS- (((1000000* (UART1_RCFIFO + GPS_Avail)*10 ) ) / GPS_BAUD );
Now I need to add another #define with an #ifdef for sharing.

Giving an Average uS error of 8 from the 'Real time GPS_serialrx_isr()' Start bit detector code - and the RCFIFO value showing a 0,1,2,3.

With that correction the extreme error is -1 to 20 uS : or within a single character not yet a Full Byte - which is the best I can get without some elaborate wait for incoming character to complete to the bit change. The Minus 1 is from rounding on the elapsedMicro time base from different points in time and code.

This also suggests the above Rx_isr() detect code is properly seeing the Start bit RISING as I thought it was, except when things are busy and the processor isn't ticking fast enough. This version is running under TeensyThreads though not switching on me as written.

Is the Teensy bothered by doing this 5 times a second? Is it just a simple memory update that the processor deals with?

detachInterrupt(digitalPinToInterrupt(GPS_SRX));
attachInterrupt(digitalPinToInterrupt(GPS_SRX), GPS_serialrx_isr, RISING);

 

[PaulStoffregen]

Is the Teensy bothered by doing this 5 times a second?

Should be fine.

 

[defragster]

Paul - Thank You for the emboldening. Looking good it completed 4177 messages without a single miss that normally would have hit in under 500 messages at 120 MHz. This is done with 100 Hz USB Serial prints of 133 chars.

Being a bit more bold I removed the TIME check used before to skip the chatter and it is running like clockwork through 1690 successful messages at 120 MHz!

Looks like I can skip the less accurate FIFO guessing game code I started this thread for, but that is working as well showing the to be expected 8 uS avg error to the Rx isr() timing - which confirms the isr() is catching it right!

Removing that spare CHATTER of triggering on rising during 1000 data bits { with minimum 99 more STOP>START plus 'data changes' at GPS_BAUD 460800 } seems to allow my more precise GPS_serialrx_isr() to run on a 120 MHz CPU without missing! As noted these messages are at 5 Hz.

And to double down I recompiled at 96 MHz on the T_3.6. I'll let that run overnight - so far 10,100+ GPS messages have been timed 1 for 1 for completed messages.

Updated as follows:

FASTRUN void GPS_serialrx_isr() {
    GPS_RX_time = tsGPS; // Start bit transition detected after 'idle' time
    detachInterrupt(digitalPinToInterrupt(GPS_SRX));
    GPS_newData=1;
    GPSrxs++;
}

And the debug GPSrxs value matches 1 for 1 with # of completed GPS messages as received. When a GPS receive is complete the attachInterrupt() is redone and ~197 ms of idle time follows waiting for the Rising START bit!

For the record - this is running under TeensyThreads - though that was a test to be skipped for now so this final version of Post #1 code shows that verbiage around the calculation of when the message started:

    while ( 0 == (GPS_Avail = GPS_PORT.available() ) ) threads.yield();
    if ( 0 == GPS_State ) {
[B]      GPS_State = tsGPS- ((1000000* ((UART1_RCFIFO + GPS_Avail)*10 ) ) / GPS_BAUD );[/B]
    }

The tsGPS is an elapsedMicros value common to both - that is how I determine the 8 uS difference in the methods and see them both working.