Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 15 of 15

Thread: Teensy 3.2 UART halts (locks up) if a Framing Error or Buffer Ovr Occurs

  1. #1

    Teensy 3.2 UART halts (locks up) if a Framing Error or Buffer Ovr Occurs

    Hi

    I'm using Teensy 3.2 and we're doing a long term project with teensy.

    We've recently come across a situation that's not handled by the Teensiduino library we use and causes de UART to stop receiving characters.

    If you refer to the processor datasheet, page 1215, you will read:

    Under the Overrun bit:
    " If the OR flag is set, no data is stored
    in the data buffer even if sufficient room exists. Additionally, while the OR flag is set, the RDRF and IDLE
    flags are blocked from asserting
    , that is, transition from an inactive to an active state. "

    For the Framing Error bit:
    " FE inhibits further data reception until it is cleared.
    To clear FE, read S1 with FE set and then read D. "

    None of these cases are treated in the serialX.c file (again in the version we use).

    Original code was (for Serial2.c)

    Code:
    #ifdef HAS_KINETISK_UART1_FIFO
    	uint32_t newhead;
    	uint8_t avail;
    
    	if (UART_S1 & (UART_S1_RDRF | UART_S1_IDLE)) {
    		__disable_irq();
    		
    		avail = UART1_RCFIFO;
    		if (avail == 0) {

    Changed to (but not yet working properly):

    Code:
    #ifdef HAS_KINETISK_UART1_FIFO
    	uint32_t newhead;
    	uint8_t avail;
    	
    	uint8_t uartreg_S1 = UART1_S1; /* read and cache */
    	
    	if (uartreg_S1 & (UART_S1_RDRF | UART_S1_IDLE /* IMPORTANT, ADDED to treat: */| UART_S1_OR | UART_S1_FE)) {
    		__disable_irq();
    		
    		/* my own internal COMM stats; these 2 lines are optional */
    		//if (uartreg_S1 & UART_S1_OR) b_serial2_bo_count++;
    		//if (uartreg_S1 & UART_S1_FE) b_serial2_fe_count++;		
    		
    		avail = UART1_RCFIFO;
    		if (avail == 0) {
    Reusing the code that was already in place for UART_S1_RDRF | UART_S1_IDLE deals with the necessary conditions: reading the UART_S1 and reading D.

    If this has alreayd been fixed in any newer version, my apologies and please disreagard this.

    If this is still an issue in the urrent Teensyduino versions, you might want to check this as the UART actually stops receiving characters. A Serial.end() and Serial.begin() are required to resume reception of characters.

    I have confirmed characters are sent using a Logic Analyser but the last one Teensy sees is the one immediately before occuring the Framing Error.
    This is not to say that Operating with Framing Errors is a great practice; but halting the UART on such occasion isn't desirable as well.

    Regards,
    Last edited by pramilo; 01-11-2018 at 08:22 PM.

  2. #2
    Let me add that the code above still doesn't fully resolve the issue;

    I've now added the interrupts for ORIE and FEIE (interrupts for both exceptions) but the UART still halts after a FE, despite the code changes applied.

    If there's any advice it's welcome.
    Last edited by pramilo; 01-11-2018 at 08:23 PM.

  3. #3
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,701
    Yikes. I've put this on my list of issues to investigate.

    Can you tell me a little more about "way off" for the stop bit? Does this mean the signal is kept held low beyond the stop bit time? How many more bit times? Or something else? In other words, if I try to create waveforms to reproduce this scenario, can you tell me what I should try to create to reproduce what you're seeing?

  4. #4
    Hi Paul,

    Thank you very much for taking the time to look into this.
    I would gladly send you a screenshot of the Logic Analyser as a picture's worth a thousand words but I'm not at the Office right now (I'm in London time).

    As described, what I found was that when sending a stream of bytes to Teensy, if one of the bytes has a Framing Error, Teensy stops receiving any further bytes.
    In the Logic Analyser I can see the stream of bytes being sent to Teensy, I can see the framing error and I can see the bytes continuing to flow.
    All bytes BEFORE the framing error are received by Teensy. All bytes after are not.

    I thought the behavior with regards to timing had had changed when I applied the code changes but I was wrong; apologies for that incorrect information. The code change I applied did not change the behaviour.

    The malfunction occurs if you simply generate a waveform of a byte transmission with a STOP bit set to LOW (in other words a Framing Error).
    A typical, correct Byte transmission is:
    [START BIT = LOW][...8 data bits HIGH/LOW doesn't matter ...][STOP BIT = HIGH]

    If you do instead:
    [START BIT = LOW][...8 data bits HIGH/LOW doesn't matter...][STOP BIT = LOW] (which is essentially a Framing Error), teensy will not receive any characters after this.

    The line can be brought high immediately after the timing of the STOP bit ends; it doesn't matter. What seems to matter is that the line is held LOW during the time of the STOP bit.

    This is what I have tried so far:

    So far referring to the datasheet of the processor, it seems expectable that the UART halts after a Framing Error as explained in the original post.
    I've tried the code change in the original post but it had no effect; the code seems to do what the Manual explains: read S1 and read D but it doesn't get the UART back to working condition.

    I also suspected that the conditions could be occurring outside the moment when the IRQ was called and thus blocking the IRQ call, so I enabled the ORIE and FEIE interrupts in the registers.
    That didn't help either but, on this particular case, I'm actually not sure if I enabled them correctly. I only set them up int he UART registers; I don't know if I need to do anything... I come from ATMEL and I'm not very well acquainted with the Interrupt architecture on Teensy.

    I will continue to work on this as I need to have the UART tolerating these anomalies. Any input from you is most welcome.

    Best Regards

  5. #5
    A Google Search (as I don't have equipment with me) has shown a few related posts:

    https://freescale.jiveon.com/thread/388930
    https://community.nxp.com/thread/312294
    (I'm a bit at a loss if this one gives any clue: ) https://community.nxp.com/thread/341862

    From reading it seems that in addition to reading S1 and clearing D, we need to reset the FIFO.
    It also seems D will have the character with the framing error according to the manual (haven't confirmed yet). This means in serialX.c, we may have UART1_RCFIFO > 1 and therefore it will run the code branch that doesn't reset the FIFO (the code it only resets the FIFO if there are no characters)

    Maybe this is what's missing. Will try and let you know.

  6. #6
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,701
    Quote Originally Posted by pramilo View Post
    I would gladly send you a screenshot of the Logic Analyser as a picture's worth a thousand words
    Yes, please post the screenshot on this thread. That would be tremendously helpful, because the very first thing I will do before I even look at the code is try to program another Teensy to create the waveform.

    Realistically, I won't be able to do much until at least next week. There are still several things I want to wrap up with the MIDI updates.

  7. #7
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,701
    I'm planning to look into this issue today. Any chance you could post that logic analyzer screenshot?

  8. #8
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,701
    My first attempt to reproduce this problem has failed. Here's what I tried.

    I connected a Teensy 3.2 to receive and a Teensy 3.6 to transmit a byte with framing error.

    Click image for larger version. 

Name:	DSC_0956_web.jpg 
Views:	25 
Size:	158.8 KB 
ID:	12593

    The Teensy 3.6 waits for high-to-low pulse on the orange wire. Then it transmits the byte 0 in 8N1 format at 8533 baud. That speed should be exactly 9 bits at 9600 baud. Here's the code runnong on the Teensy 3.6:

    Code:
    //const int baud=9600;
    const int baud=8533; // 9600 * 9/8 (9 bit times)
    
    void setup() {
      Serial1.begin(baud);
      pinMode(2, INPUT_PULLUP);
    }
    
    void loop() {
      if (digitalRead(2) == LOW) {
        Serial1.write(0);
        while (digitalRead(2) == LOW) /* wait */ ;
        delay(10);
      }
    }
    I connected my oscilloscope to verify it really is a framing error at 9600 baud.

    Click image for larger version. 

Name:	file.png 
Views:	26 
Size:	26.7 KB 
ID:	12594

    On the Teensy 3.2, I wrote a very simple program to just receive each incoming byte and print to the serial monitor. If things lock up, we should get only 1 line printed? Or no lines?

    Code:
    unsigned int count=0;
    
    void setup() {
      Serial1.begin(9600);
    }
    
    void loop() {
      if (Serial1.available()) {
        int n = Serial1.read();
        Serial.printf("Received %u, byte count=%u\n", n, ++count);
      }
    }
    When I run this and tap the orange wire to ground many times, here's the results I see. No sign of any lockup.

    Click image for larger version. 

Name:	sc.jpg 
Views:	27 
Size:	143.5 KB 
ID:	12595

    I tested with Arduino 1.9.0-beta31 and Teensyduino 1.41-beta3 (plus some MIDI updates, which shouldn't matter for this), using Linux 64 bit (Ubuntu 14.04).

  9. #9
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,701
    I tried just now to investigate the overflow condition. Again, I could not reproduce the problem.

    I used the same hardware as above. The Teensy 3.6 sends 12 bytes when the orange wire pulses low.

    Code:
    void setup() {
      Serial1.begin(9600);
      pinMode(2, INPUT_PULLUP);
    }
    
    void loop() {
      if (digitalRead(2) == LOW) {
        for (int i=1; i <= 12; i++) {
          Serial1.write(i + 32);
        }
        delay(10);
        while (digitalRead(2) == LOW) /* wait */ ;
        delay(30);
      }
    }
    The Teensy 3.2 turns off interrupts and gives a 15 ms pulse (rather than my hand touching the wire to GND), which is long enough for all 12 bytes to transmit. Interrupts remain disabled for the entire 15 ms. Here is are the waveforms on my scope.

    Click image for larger version. 

Name:	file.png 
Views:	29 
Size:	31.6 KB 
ID:	12597

    It's easy to see on the scope that all 12 bytes really are transmitting.

    Here is the code running on the Teensy 3.2. After the 15 ms pulse, it turns interrupts back on and attempts to read whatever data arrived.

    Code:
    unsigned int loopcount=0;
    
    void setup() {
      Serial1.begin(9600);
      pinMode(12, OUTPUT);
      digitalWrite(12, HIGH);
    }
    
    void loop() {
      unsigned int count=0;
      
      noInterrupts();
      digitalWriteFast(12, LOW);
      delayMicroseconds(15000);
      digitalWriteFast(12, HIGH);
      interrupts();
      while (Serial1.available()) {
        int n = Serial1.read();
        Serial.printf("Received %u, ", n);
        Serial.printf("byte count=%u, ", ++count);
        Serial.printf("loop count=%u\n", loopcount);
      }
      loopcount++;
      delay(1000);
    }
    As you can see in this screenshot, it's not locking up.

    Click image for larger version. 

Name:	sc.jpg 
Views:	25 
Size:	163.1 KB 
ID:	12598

    As expected, only the first 8 bytes are received because Serial1 has a 8-deep FIFO. The other 4 bytes are lost, which should be pretty good confirmation of the overflow condition. But no lockup is happening. In fact, while I've been writing this message, it's continued running, now with loopcount up to 778 and still running without any lockups.

  10. #10
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,701
    Normally I only investigate problems when complete code and clear details on the test process are posted. In this case I made an exception, because the nature of this problem could possibly be severe, and because 1.41 is due to release soon.

    At this point, I'm going to consider this a non-issue. I'm not necessarily saying Teensyduino's code is perfect. There very well may be a problem. But if you want me to look into this further, you really must post complete programs and details on how to reproduce the problem. Hopefully these 2 messages can serve as a sort of template on the level of detail needed.

    Please verify that your program and test steps actually reproduce the problem with 1.41-beta3, or whatever happens to be the most recently beta or full release. There's no point to wasting time with testing old versions.

    I do take these sorts of issues very seriously. If there is a problem, I want to find and fix it. But without complete code and detailed instructions to reproduce the issue, and with my best attempts seeming to confirm there isn't any issue here, I just put any more time into this one.

  11. #11
    Quote Originally Posted by PaulStoffregen View Post
    Yes, please post the screenshot on this thread. That would be tremendously helpful, because the very first thing I will do before I even look at the code is try to program another Teensy to create the waveform.

    Realistically, I won't be able to do much until at least next week. There are still several things I want to wrap up with the MIDI updates.
    Hi Paul,

    I missed your other posts after this one as I was not in the office Friday and basing myself on your timing.

    In any case, I have the screenshot now.

    Feel free to address this as you feel more appropriate or even close it as in a ticket system, in case it no longer applies..
    I felt I should point this out nevertheless and will update this if we find anything else.

    There are two other conditions worth pointing:
    - We are using the UART in one WIRE mode using the native one wire capabilities in the chip (LOOPs)
    - I can't figure out which version fo Teensyduino I have. If there's any way to check this I'd appreciate the advice on how to do it.
    I can post the source code for the SerialX.c we're using but we won't be upgrading to any newer teensyduino version as it breaks too much stuff where we placed any kind of customization. We'll pursue the fix ourselves.

    Click image for larger version. 

Name:	FramingError.PNG 
Views:	39 
Size:	6.4 KB 
ID:	12614

    This is the screenshot.

    EDIT: one thing to note is that you can see extenal interference while this char is trasmitted (some fast peaks while the line is low). Not sure if it also contributes to the problem at hand. The actual char is a '0' as the logic analyser is identifying.

    I can get the chars up to the last 0. The character afterwards (and all other characters after) don't have any framing errors but won't be received.
    Last edited by pramilo; 01-15-2018 at 06:18 PM.

  12. #12
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    3,706
    Quote Originally Posted by pramilo View Post
    There are two other conditions worth pointing:
    - We are using the UART in one WIRE mode using the native one wire capabilities in the chip (LOOPs)
    - I can't figure out which version fo Teensyduino I have. If there's any way to check this I'd appreciate the advice on how to do it.
    I can post the source code for the SerialX.c we're using but we won't be upgrading to any newer teensyduino version as it breaks too much stuff where we placed any kind of customization. We'll pursue the fix ourselves.
    If you are running using Arduino IDE. You can go to the Help->About Arduino command and the popup display will show you which version of arduino as well as the version of Teensyduino.

    Also if you do a Verbose Compile (Set in Arduino preferences) and look at the command lines generated, you will see some defines in the compiles that include things like: -fsingle-precision-constant -D__MK66FX1M0__ -DTEENSYDUINO=141 -DARDUINO=10900

    So for my current compile you can see I was compiling for Teeny 3.6 I have Teensyduino 1.41 (latest beta) and I am running Teensyduino 1.9 (in this case beta 34)...

  13. #13
    Thank you for the advice.
    I will look into your advice and post more information.

  14. #14
    I have now been able to get a more clear undersatding of what is gong on in my case.

    Before I continue I have determined my versions: Arduino 1.0.6 and Teensyduino 1.25. I know these date far back but looking at newer code it seems serialX adds support for newer processors but doesn't rewrite the way Buffer Overflow or Framing Errors are handled.

    Again, I am posting this just in the spirit of sharing the experience. Feel free to ignore it.

    Below is an overviwe of a full transmission of 2 blocks of data. Both blocks are the same data.
    However on the first Block there is a Framing error on the 3rd character (the second '0').

    IMPORTANT: each blocks is sent in a burst, with no delay or interval between them.
    I have debugged the serial library and the interrupts keep firing with at least 4 characters in the FIFO, while the burst is being transferred and until it finishes.

    The overall view in terms of timing, for the two blocks is:
    Click image for larger version. 

Name:	Capture.PNG 
Views:	25 
Size:	10.6 KB 
ID:	12618

    Zooming in the on the first block:
    Click image for larger version. 

Name:	detail1.PNG 
Views:	30 
Size:	8.5 KB 
ID:	12616

    You will see the framing error. If you compare the timing with the overall screen shot I posted first, you can locate it.
    Data after the first framing error is not seen by teensy. Ever.
    I tried multiple combinations of clearing the FIFO buffer, disabling and re enabling the FIFO; nothing seemed to work.

    HOWEVER what I have found is that after the IDLE period between Block 1 and 2, when I re-transmit the data, the second time without any framing errors, Teensy is receiving data again and will receive this data correctly.
    This is the Zoom in for the re-transmission:
    Click image for larger version. 

Name:	detail2.PNG 
Views:	27 
Size:	11.2 KB 
ID:	12617
    Notice there is no framing error.

    It sems that between block 1 and block 2, during that IDLE period, teensy recovers. I'm inclined to think this is processor related (not Teensyduino related).
    I don't know why it only recovers after the IDLE period and not immediately after the Framing Error.

    I also haven't done any tests regarding the IDLE period necessary for the recovery. In this case I waited fairly long before re transmitting as you can see from the pictures.

    For me this is sufficient. Knowing this will let me get the things I need working. If anyone else runs into this, well, feel free to use as you see fit.

    My apologies to Paul, as my headline in the OP is indeed innacurate. Teensy does recover from FE and BO. In my case, it will do it after an IDLE period and will lose/miss data before the IDLE period.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	overall.PNG 
Views:	23 
Size:	3.8 KB 
ID:	12615  
    Last edited by pramilo; 01-15-2018 at 09:55 PM. Reason: Trimmed the image of the overview to show only the issue at hand

  15. #15
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,701
    Quote Originally Posted by pramilo View Post
    my versions: Arduino 1.0.6 and Teensyduino 1.25.
    Any chance you could confirm with 1.41-beta4 (the latest code)?

    If using Windows, just download the ZIP file copy of Arduino 1.8.5. Don't run the .EXE installer, since it will try to uninstall the old copy. The ZIP files (or .tar.xz on Linux) are stand-alone copies you can extract anywhere on your computer. Just remember where you put it, since it'll need to select it with the Teensyduino installer.

    The installer only modifies the copy of Arduino you select, so you can safely test with a copy of Arduino 1.8.5 and install the latest Teensyduino into that separate copy, without interfering with the old version you've been using.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •