Yes, indeed. Isochronous rate feedback used for asynchronous mode audio uses a 3 byte format when running at 12 Mbit speed, and a different 4 byte format for 480 Mbit speed. Frame timing is also different, 1 ms at 12 Mbit versus 125 us at 480 Mbit.
Currently all the existing USB audio code is written for the USB controller in the older chips and designed only for 12 Mbit speed.