Yes, to all your questions, except "sort of" on the library status. The core EHCI & enumeration stuff is pretty solid, and the serial and hub drivers work very well. The library still does not support isochronous at all, and there's a long list of drivers to make, so large changes can be expected to the library for quite some time. There's also a good chance we'll eventually migrate to using EventResponder for some stuff, which could be considered quite a large change.
Support for high-speed (480 Mbit) USB serial devices is still lacking. Not an issue with Teensy 3.x boards.
Of course the available memory does limit the number of driver instances. Teensy 3.6 has quite a lot of memory and each instance doesn't use much memory, but eventually if you expand to hundreds of devices, you will hit memory limits.