Mac OS X El Capitan Serial port problems - need more info to raise issue with Apple

Status
Not open for further replies.

howiemnet

Active member
I've been having ongoing issues with serial comms with Teensy 3.1/LCs, and have finally tracked it down to the El Capitan betas I've been running on my Macs. The behaviour is strange, and I'd like to raise it with Apple, but I'd like as much info as possible about the Teensy's USB-Serial method as possible so I can try and Make It Easy For Them To Understand. They may not know what a Teensy is. It would be very helpful if I could, say, find another device that exhibits the same issues.

The problem:

Simple loopback code on a Teensy (I've tested LC and 3.1):

Code:
void setup() {
  Serial.begin(115200);
  pinMode(13,OUTPUT);
}

void loop() {
  if (Serial.available() > 0) {
    digitalWriteFast(13, HIGH);
    Serial.write(Serial.read());
    digitalWriteFast(13,LOW);
  }
}

Whatever the application I use to talk to the Teensys (I'm writing my own, but this happens with any serial terminal software (eg CoolTerm), and even happens with the Serial Monitor in the Arduino IDE), I get the following behaviour cycle:

- open the serial port: everything works perfectly. Type on the Mac and it gets echoed back.
- close the serial port: whatever the software / terminal package I'm using, this seems to tie the application up for several seconds
- open the serial port again: no response from the Teensy. Chars are transmitted OK but ignored by the Teensy.
- close the serial port again: port closes instantly.
- back to the start of this cycle - everything'll work OK again temporarily...

This problem does not occur if I run the same code on an Arduino Mega.

What technology is behind the USB-Serial interface provided by the Teensy? It doesn't need a driver installed, but it operates super-fast - so how does it work? Does it pretend to be an FTDI or Prolific chip?

Are there any other dev boards or devices that use the same method of comms, so I can check that this problem is consistent?
 
... in fact this can all be reduced to this:

- on a Mac running any version of El Capitan
- upload the loopback code above (or write any code that uses the Serial port)
- switch on and off the Serial Monitor in the Arduino IDE and observe the behaviour
 
Does an output-only sketch reproduce the problem?

Maybe this?

Code:
int ledpin = 13;

void setup() {
  Serial.begin(9600);
  Serial1.begin(115200);
  pinMode(ledpin, OUTPUT);
}

int count = 0;

void loop() {
  Serial.print(" count = ");
  Serial.println(count++);
  Serial1.print(" count = ");
  Serial1.println(count);
  digitalWrite(ledpin, HIGH);
  delay(150);
  digitalWrite(ledpin, LOW);
  delay(150);
}
 
Yes, problem still there, but the flashing LED may provide clues. Behaviour this time (pretty much as before):

After flashing the output-only code to a Teensy 3.1:
- plug into a Mac - the LED starts flashing straight away
- open the serial monitor - count appears - 1,2,3,4,5,6,7,8,9,10, then jumps to 310,311 etc (presumably just the buffer got full)
- close the serial monitor - the LED stops flashing, the Arduino IDE freezes for a few seconds
- open the serial monitor - nothing - no text appears. The Teensy seems to have stopped.
- close the serial monitor - The Teensy's LED starts flashing again.

Next time you open the monitor we'll be back at it working again. Let me know if you want me to pull an LC to test but I suspect it'll be similar.
 
I've put this at the top of my list of issues to investigate. May be a couple days, but I do intend to get to this one soon! (hopefully without an expensive trip to an Apple store....)
 
Anything I can do to help, let me know. My current project is probably going to become my career (!) so I'm sitting here poised to kick Apple as soon as I can form a lucid and as-generalised-as-possible bug report for them :)

At least you don't need recent hardware -- it was my oldest Macbook (a battered old Core 2 Duo) that confirmed the issue was El Capitan - it was the only machine I could afford to wipe and do a fresh install of OS X on. I stuck Mountain Lion on it and upgraded it all the way to El Capitan, testing the serial behaviour with each step.
 
More information: This problem affects uploading sketches via the Arduino IDE -> Teensy Loader too. Every other time the Serial port is opened, the IDE/Loader will fail to upload automatically and you'll have to press the button. Once flashed, the Teensy is working, meaning:

- If you open and close the Serial Monitor an odd number of times before trying to upload, it'll fail to upload (so hit the button)
- If you open and close the Serial Monitor an even number of times before trying to upload, it'll work automatically

Everything looks like the Teensy itself hanging when trying to close a working serial port from the Mac end. Open the port again - dead Teensy, close the port again, and the Teensy springs back to life.
 
What happens if you close arduino and the teensy loader and do the open-close with a terminal-program only ?
 
You still get the problem, whatever the software. I've been using my own app to handle serial comms and assumed for some time that it was something I was doing wrong. And that's not off the table - but I'm reasonably confident I haven't buggered the Arduino IDE up or done something stoopid to my Teensys (I always remember to boil them in water for 5 minutes after flashing new code to make sure it "takes", just like my dear old grandma showed me)

Half relieved, half worried now I seem to have identified OS X as the culprit - if I'm right, then it's something Apple have changed, and it could be some time before they pull their finger out their back end and fix it.

Unless, of course, I can show them it affects lots of other devices using the same USB<>Serial technology. Have to admit I'm kinda worried I can't find anyone else talking about this problem ...
 
Having a fresh install of OS X El Capitan, I can confirm that the issue persists with the final release.

I get the following in the system log every time the teensy hangs, although Bluetooth is disabled on my MacBook Pro:
04/10/2015 13:49:18,000 kernel[0]: **** [IOBluetoothHostControllerUSBTransport][InterruptReadHandler] -- Received kIOReturnNotResponding error - retrying: 1
 
Yes, problem still there, but the flashing LED may provide clues. Behaviour this time (pretty much as before):

After flashing the output-only code to a Teensy 3.1:
- plug into a Mac - the LED starts flashing straight away
- open the serial monitor - count appears - 1,2,3,4,5,6,7,8,9,10, then jumps to 310,311 etc (presumably just the buffer got full)
- close the serial monitor - the LED stops flashing, the Arduino IDE freezes for a few seconds
- open the serial monitor - nothing - no text appears. The Teensy seems to have stopped.
- close the serial monitor - The Teensy's LED starts flashing again.

Next time you open the monitor we'll be back at it working again. Let me know if you want me to pull an LC to test but I suspect it'll be similar.

Just had some minutes to test it (MacBook Air)
Exactly the same behavior.
Interestingly, the count starts with 1 again when the monitor opens. Does the mac switch off the power or resets the Teensy ?
 
So, to distil: closing the serial port hangs the Teensy before it's had a chance to completely "disconnect", causing the host Mac to wait for a timeout. Opening the port again leaves you connected to the hung Teensy, but closing it a second time resets the Teensy.

Uploading a sketch while the Serial Monitor is open will fail; you can reach over and hit the Teensy's reset button, or if it's not easy to reach I've found yanking the USB lead out, then closing the Serial Monitor, then replugging, then clicking upload works the fastest. (Don't bother trying to close the Serial Monitor before yanking the cable - it takes longer that way. Have to close it before replugging in the Teensy or it'll automatically re-open the port again, d'oh)

Dagnammit, what have you changed, Apple? I'm stuck in limbo between two operating system versions (my app needs El Capitan, my Teensys need Yosemite) and am wearing out my USB sockets swapping between machines every 5 minutes arrrrrggggg I need a cuppa tea and a sit down arrrgg
 
koromix made this way cool tool TYQT that makes the Teensy user experience better for me on Windows with multiple Teensy's - or just a good replacement for IDE serial mon.

He tests the least on apple - though he sees downloads about half of those for Windows and way more than Linux - but expects it to work and would welcome feedback I expect if it has issues.

Read the thread but when it is SerMon attached it can perform RESET or trigger Teensy to UPLOAD. You have to close the IDE SerMon and may need to restart the Teensy for it to see it. But in my experience once it does it offers great functionality that doesn't interfere with TeensyLoader - and the UPLOAD and RESET work to coordinate the USB dropping so TeensyLoader can do it's job and then should see the USB come back online to Monitor for you. YMMV may vary on apple - or if your app is looking for the USB to be available - you'll need to close TYQT for that to happen. A couple times I've wished for TYQT to have a 'disconnect' USB to device - I'll go post that as a feature request, then you could leave TYQT running and not have it tie the Teensy down - when you have other Teensy's online.

NOTE: Teensy does not ship with a reset button - its button just takes the unit offline to program mode.

NOTE: There is some odd TeensyLoader behavior I've not quantified - but it takes overt action to reset some STATE it gets into.
Action: Close TeensyLoader, Hit Verify in IDE - that freshens the compile and restarts TeensyLoader.
{perhaps hit 'program button' after Teensy closed, or with TYQT hit RESET }
 
@defragster - wow, TYQT looks useful - I'll have to give it a go. Can't count the number of times I've accidentally flashed the wrong Teensy by mistake. My current rig has 8 Teensys connected (which I know sounds ridiculous, but it actually works really well and lets me keep things modular and manageable)

@DonLakeFlyer - Useful to know, many thanks. Have to see if we can work out what the Pixhawks have in common with the Teensy... Cheers :)
 
I tested on a Mac with El Capitan and I was able to reproduce the problem. (also mentioned here)

It's due to a bug in handling endpoint halting, deep in the Teensyduino USB stack. Until El Capitan, none of the operating systems have used this, so the bug went unnoticed all this time.

Well, actually, I noticed it a couple weeks ago by random chance while working on the MTP stuff. I fixed the bug as part of this initial MTP work. Here's the specific fix:

https://github.com/PaulStoffregen/c...117#diff-7b3a44510f0e4eee93c6b42a56b5b3b5L279

I just tested this with El Capitan. It does solve the problem.

If you'd like to try this now, right click (or control click) on Arduino and choose "Show Package Contents". Then navigate to Contents/Java/hardware/teensy/avr/cores/teensy3. You'll need to replace everything in that folder with the contents of this github repository. Use "Download ZIP" on the right hand side to download the whole thing.

https://github.com/PaulStoffregen/cores

Within the next couple weeks I'm going to put together a first beta for 1.26, which will have this fix.

I'm also going to try to find way to create a Yosemite and El Capitan compatible signatures without losing Snow Leopard and Lion compatibility. Unforunately, that's a *lot* harder than fixing USB bugs! Apple really makes it tough to build compatibly for their older systems, so I'm going to try to set up a build process using two macs....
 
Are there any more threads about this issue? That other one was the link I saved on my TODO list. The point now it only to notify everyone who may be waiting for this fix.
 
Status
Not open for further replies.
Back
Top