Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 8 of 8

Thread: Question about UART FIFO containing one byte and when RDRF gets set (one char behind)

  1. #1
    Senior Member
    Join Date
    Mar 2017
    Location
    Oakland, CA, USA
    Posts
    119

    Question about UART FIFO containing one byte and when RDRF gets set (one char behind)

    When I need to measure character timestamps from a UART (having a FIFO), I use the occurrence of RDRF as the time. This has been working well, but I noticed that when I receive only a single character, the RDRF flag gets triggered about one character behind when the character was received. In other words, measuring `micros()` is about one character time too late. (This makes sense because this is, after all, a buffer.)

    I accommodate this with the following logic in the ISR (complete program):
    Code:
    #include <kinetis.h>
    
    void setup() {
      Serial.begin(500000);
      while (!Serial) {}
    
      pinMode(LED_BUILTIN, OUTPUT);
    
      Serial1.begin(250000, SERIAL_8N2);
      attachInterruptVector(IRQ_UART0_STATUS, uart0_rx_isr);
      while (!Serial1) {}
    
      Serial2.begin(250000, SERIAL_8N2);
      while (!Serial2) {}
    }
    
    void loop() {
      static elapsedMillis p{1000};
      static bool mode = false;
      static uint8_t b[2]{0x02, 0x03};
    
      // Every second, Serial2 sends data to Serial1, alternating between
      // one byte and two bytes
      if (p >= 1000) {
        if (!mode) {
          Serial2.write(0x01);
          Serial2.flush();
          Serial.printf("tx: %d: %02xh\n", micros(), 0x01);
        } else {
          Serial2.write(b, 2);
          Serial2.flush();
          Serial.printf("tx: %d: %02xh %02xh\n", micros(), b[0], b[1]);
        }
        digitalWrite(LED_BUILTIN, HIGH);
        digitalWrite(LED_BUILTIN, LOW);
        p = 0;
        mode = !mode;
      }
    }
    
    // Handles a byte. The timestamp marks the end of the character.
    void handleByte(uint8_t b, uint32_t t) {
      Serial.printf("RX: %d: %02xh\n", t, b);
    }
    
    // Handles UART0 receive.
    void uart0_rx_isr() {
      if ((UART0_S1 & (UART_S1_RDRF | UART_S1_IDLE)) != 0) {
        __disable_irq();
        uint8_t avail = UART0_RCFIFO;
        if (avail == 0) {
          // Re-align the FIFO
          UART0_D;
          UART0_CFIFO = UART_CFIFO_RXFLUSH;
          __enable_irq();
        } else if (avail == 1 && UART0_RWFIFO > 1) {
          __enable_irq();
          // It appears that the data-available flag isn't triggered until
          // one character time after there is only one character available,
          // so accommodate this when the FIFO has only one element
          UART0_S1;
          handleByte(UART0_D, micros() - 44);  // <-- HERE
        } else if (avail > 0) {
          __enable_irq();
          uint32_t timestamp = micros() - 44*avail;
          while (--avail > 0) {
            handleByte(UART0_D, timestamp += 44);
          }
          UART0_S1;
          handleByte(UART0_D, timestamp += 44);
        }
      }
    }
    Results:
    Code:
    tx: 1934097: 01h
    RX: 1934096: 01h
    tx: 2934097: 02h 03h
    RX: 2934096: 02h
    RX: 2934140: 03h
    tx: 3934049: 01h
    RX: 3934048: 01h
    tx: 4934097: 02h 03h
    RX: 4934096: 02h
    RX: 4934140: 03h
    tx: 5934049: 01h
    RX: 5934048: 01h
    tx: 6934097: 02h 03h
    RX: 6934096: 02h
    RX: 6934140: 03h
    Notice that the characters are received at approximately the same time as they are sent, with the second byte of the 2-byte send being received 44us after the first byte. The point to note here is that for the 1-byte send, I need to subtract 44 before passing it to `handleByte` in order to get the correct character-end timestamp. This means that there is a one character delay before the RDRF bit is triggered.

    To try this yourself, connect Serial2 TX to Serial1 RX.

    My questions:
    1. Does this only happen when UARTx_RWFIFO > 1? i.e. do I need that extra logic or is 'avail == 1' sufficient?
    2. Is "one character behind" the only case, or will this ever be "N characters behind"?
    3. I can't seem to find a mention of this in any of the KINETISK chip specs (for any of the Teensy 3's); does anyone know where this might be mentioned?
    4. Does the Teensy 4 processor have this same behaviour, according to its chip docs? Again, I couldn't find a mention anywhere.


    I mean, this makes sense when having a FIFO because the point is to buffer, but I'm trying to find where this is mentioned and how this relates to the watermark.
    Last edited by shawn; 12-04-2019 at 10:32 PM. Reason: Title adjustment and one more question

  2. #2
    Senior Member
    Join Date
    Mar 2017
    Location
    Oakland, CA, USA
    Posts
    119
    Ok, I got smarter about this and instead of special-casing `avail==1`, I'm comparing a positive value for `avail` with the watermark size. If it's smaller, then one character time is subtracted. Below is updated code, also updated to work with a Teensy 4.

    Code:
    #if defined(__IMXRT1062__)
    #include <imxrt.h>
    #else
    #include <kinetis.h>
    #endif
    
    void setup() {
      Serial.begin(500000);
      while (!Serial) {}
    
      pinMode(LED_BUILTIN, OUTPUT);
    
      Serial1.begin(250000, SERIAL_8N2);
    #if defined(__IMXRT1062__)
      attachInterruptVector(IRQ_LPUART6, lpuart6_rx_isr);
    #else
      attachInterruptVector(IRQ_UART0_STATUS, uart0_rx_isr);
    #endif
      while (!Serial1) {}
    
      Serial2.begin(250000, SERIAL_8N2);
      while (!Serial2) {}
    }
    
    void loop() {
      static elapsedMillis p{1000};
      static bool mode = false;
      static uint8_t b[2]{0x02, 0x03};
    
      // Every second, Serial2 sends data to Serial1, alternating between
      // one byte and two bytes
      if (p >= 1000) {
        if (!mode) {
          Serial2.write(0x01);
          Serial2.flush();
          Serial.printf("tx: %d: %02xh\n", micros(), 0x01);
        } else {
          Serial2.write(b, 2);
          Serial2.flush();
          Serial.printf("tx: %d: %02xh %02xh\n", micros(), b[0], b[1]);
        }
        digitalWrite(LED_BUILTIN, HIGH);
        delay(20);
        digitalWrite(LED_BUILTIN, LOW);
        p = 0;
        mode = !mode;
      }
    }
    
    // Handles a byte. The timestamp marks the end of the character.
    void handleByte(uint8_t b, uint32_t t) {
      Serial.printf("RX: %d: %02xh\n", t, b);
    }
    
    #if defined(__IMXRT1062__)
    // Handles LPUART6 receive.
    void lpuart6_rx_isr() {
      if ((LPUART6_STAT & (LPUART_STAT_RDRF | LPUART_STAT_IDLE)) != 0) {
        uint8_t avail = (LPUART6_WATER >> 24) & 0x07;
        if (avail == 0) {
          LPUART6_STAT |= LPUART_STAT_IDLE;
        } else {
          uint32_t timestamp = micros() - 44*avail;
          if (avail < ((LPUART6_WATER >> 16) & 0x03)) {
            timestamp -= 44;  // <-- HERE
          }
          while (avail-- > 0) {
            handleByte(LPUART6_DATA, timestamp += 44);
          }
        }
      }
    }
    #else
    // Handles UART0 receive.
    void uart0_rx_isr() {
      if ((UART0_S1 & (UART_S1_RDRF | UART_S1_IDLE)) != 0) {
        __disable_irq();
        uint8_t avail = UART0_RCFIFO;
        if (avail == 0) {
          // Re-align the FIFO
          UART0_D;
          UART0_CFIFO = UART_CFIFO_RXFLUSH;
          __enable_irq();
        } else {
          __enable_irq();
          uint32_t timestamp = micros() - 44*avail;
          if (avail < UART0_RWFIFO) {
            timestamp -= 44;  // <-- HERE
          }
          while (--avail > 0) {
            handleByte(UART0_D, timestamp += 44);
          }
          UART0_S1;
          handleByte(UART0_D, timestamp += 44);
        }
      }
    }
    #endif
    Maybe this will be useful to some. See the "HERE" lines in the code.

    It's still a question to me whether this "delayed case" will always be only one character behind, or if it's possible to be more characters behind.
    Last edited by shawn; 12-05-2019 at 01:13 AM.

  3. #3
    Senior Member
    Join Date
    Mar 2017
    Location
    Oakland, CA, USA
    Posts
    119
    What this is for: Measuring where packets of data start and end for my TeensyDMX library (see the latest `teensy4` branch for all the latest stuff). I'm trying to do accurate measurements for both informational purposes (connected software that needs it) and for better timeout checking, and so far this is working. I had just encountered timings that were off by exactly one character length (44us), so started to look into this. In testing, the subtract-one-char-length-if-available-bytes-less-than-watermark approach seems to be working.

  4. #4
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    10,093
    I've posted code that puts an interrupt on the Rx pin that triggers when the STOP bit goes to START - the interrupt records the time, and turns itself off so data bit transitions are not trapped. Then the receive code can keep track of the last byte received most importantly to turn the UART_isr back on, but in this case if the reading is done as the last byte arrives that would end the time in some fashion.

    Seems I posted this code again recently - but it was originally done for GPS message arrival timing on the uNav thread : uNav-AHRS - this is one post >> uNav-AHRS - may be better posts - but that shows the guts if it is useful.
    Last edited by defragster; 12-06-2019 at 05:25 PM. Reason: UART_isr

  5. #5
    Senior Member
    Join Date
    Mar 2017
    Location
    Oakland, CA, USA
    Posts
    119
    It turns out that a lot of decent timing information can be gleaned with just the UART signals. (Verified with a scope and comparing with ostensibly-measured values.) I was avoiding using a separate pin (I'll call it a "watch pin") for a while and tried to take it as far as I could. The only thing that can't be determined when not using a watch pin, for a DMX packet, is the low-to-high transition of a BREAK to MAB. Well, that and proper timing of FIFO-buffered bytes, it turns out.

    I implemented that watch pin technique for the BREAK and MAB boundary, and when watching for both FALL and RISE (along with the disabling-the-interrupt-on-receive technique), it turned out that the logic for catching "all possible cases" for re-adding the FALL trigger turned out to be much more complicated and harder to reason about (at least for my current code structure) than the logic for just watching for the RISE. So that's what I did, I'm just watching the RISE when I see a framing error, and when that gets triggered, it marks the time and then removes itself.

    Why I think FALL checking doesn't work as well for my use case: if I wait for a received byte and then immediately attach a FALL interrupt, then in theory, I could miss any FALL that happens between when I'm notified that there's a byte and when I attach the interrupt. This especially won't work when the FIFO is in use and any byte notifications come in at least one character time late.

    So there, essentially, are two cases when the late FIFO timing comes into play: this potentially rare case when using the watch pin and when the FALL interrupt is reattached, and when I just want to figure out when characters arrived when I'm not using the watch pin.

    So if I can solve the problem of knowing when FIFO bytes actually come in vs. when I get notified that they're there, I'll consider the problem solved. The logic is simpler and I'm considering the small delay between when a character comes in and when I get notified, assuming it happens shortly after, to be negligible and acceptable.

    I think that the FIFO triggers an RDRF signal one character behind if no more bytes come in and the FIFO count is less than the watermark. If this is true, then what I've done is correct and I'm happy with the results.

    Whew! Happy to discuss further if anything I'm saying isn't making sense or if you have more suggestions.

  6. #6
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    10,093
    Setting up the 'Rx' as a watch pin served the desired purpose of seeing the Start of data bits in that case, so no need for low level UART operation was needed in that code. And with only 5 or 10 messages per second there was a clear END in a few ms with a long wait to enable the watch _isr without trouble or added machinations.

    Leaving the _isr enabled in any fashion will trigger on the data bits after that initial 'control' signal transition, so it becomes a distraction of no value.

    Not following exactly what process and need is after that initial Start - but if you get anything working on timing and can show a sample it might be good for reference. I started looking inside the UART when it became apparent the STOP>START trigger was all that was needed.

  7. #7
    Senior Member
    Join Date
    Mar 2017
    Location
    Oakland, CA, USA
    Posts
    119
    Yes, I agree that it is indeed a good idea detach the interrupt on the RX watch pin when it's triggered, so it's not triggered on all the other data transitions.

    Side question: You said: "Rx as a watch pin". Do you mean that you're triggering on FALL or RISE on the RX pin itself or on some different pin? I'm using a different pin because while I've found that it's possible to attach an interrupt at the same time as the UART is running on a Teensy 3, it doesn't work on a Teensy 4.

    Here is my solution, in pseudo-code summary form. It works, but I'm not 100% sure that it will always work. Quantitatively, it always seems to work on both a Teensy 3 and Teensy 4:
    Code:
    Notification of a byte available {
      avail = available bytes in FIFO, even if no FIFO and just one element
      timestamp = micros() - avail*kCharWidth;  // 44us for 250kbaud@8N2
      if (avail < WATERMARK) {  // <-- THIS IS THE KEY STEP
        // Assume any bytes appearing in the FIFO will notify one character time late
        // if the count is less than the watermark
        timestamp -= kCharWidth;
      }
      handleByte(DATA, timestamp += kCharWidth);
    }
    
    function handleByte(byte, end-of-byte-timestamp);
    The key point is that if characters arrive and the number of characters in the FIFO is less than the watermark, it is assumed that we've been notified one character time too late.

  8. #8
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    10,093
    The code in the link above :: pjrc.com/threads/48450-uNav-AHRS?p=168449&viewfull=1#post168449

    Puts the : attachInterrupt(digitalPinToInterrupt(GPS_SRX), GPS_serialrx_isr, RISING); - Directly on the Rx pin and it does not interfere with the GPS Serial UART comms as done.

    So the watch pin is the self same UART Rx pin. Enabled after the line goes IDLE, the RISING trigger when the last STOP bit state turns into the START bit on the next start of transfer.

    That thread likely has full code posting IIRC - that post was just the quick summary showing the _isr doing disable - then the code that re-enables it after the GOS message is read as complete.

    Though writing that I recall that STOP is HIGH and it FALLS on start which wiki seems to confirm - along with what I saw during T4 Beta when connected IDLE UARTS would give power to the low power segment of the 1062 when it was powered off - acting like an RTC battery was installed:
    The idle, no data state is high-voltage, or powered. This is a historic legacy from telegraphy, in which the line is held high to show that the line and transmitter are not damaged. Each character is framed as a logic low start bit, data bits, possibly a parity bit and one or more stop bits. In most applications the least significant data bit (the one on the left in this diagram) is transmitted first, but there are exceptions (such as the IBM 2741 printing terminal).
    Also for best resolution when it is used - the ARM Cycle Counter can get closer than micros - though micros() is built off of that cycle counter on T4 - a T4 at 600 MHz would resolve to which part of a us. But that requires proper reference and scaling with the quick rollover of the cycle counter at that speed.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •