I would exclude this.
I'm running 4.0 at 24 MHz with USB just fine (Teensyduino 1.52beta4).