Hello, I want to overclock the Teensy 3.5 above 168MHz (if possible).
I've seen some threads that mention that it could be possible, but in the Arduino IDE I do not have the option to choose anything above 168 MHz.
Does anybody know how to do it, or if it's possible?
If you can get the PLL to run faster, then turn your attention to the SIM_CLKDIV1 and SIM_CLKDIV2 to set the speed of the peripheral and flash memory. The flash memory is the part of the chip with least overclocking ability. You may wish to edit F_BUS so code controlling the peripherals "knows" the new F_BUS speed. Then you can at least get the serial ports to have correct baud rate.
USB is probably the last part you'll try, if you get that other stuff working. While fiddling, you'll probably use Serial1.print() and a USB-Serial cable, and plan on pressing the pushbutton on your Teensy 3.5 to load new code until you get USB working again. USB needs a 48 MHz clock. It can come from either the main clocks, or "IRC48M". Search usb_dev.c and the reference manual for "IRC48M" if you can't configure a 48 MHz clock for USB and want to try IRC48M.
This isn't easy. If it were easy (or even not so easy) and had substantial chance of working, it would have already been put into the code. 168 MHz is already pushing that old chip pretty far past its rated 120 MHz speed.
If you need higher performance, just buying a Teensy 4.1 is the easy path. The CPU runs at 600 MHz, but it is more than just 5X faster than 120 MHz Teensy 3.5. The M7 processor in Teensy 4.x is able to execute 2 instructions per cycle. It also has features like branch prediction, tightly coupled memory and large caches which greatly improve performance (Teensy 3.6 has a small cache which is the reason it outperforms Teensy 3.5 even at the same frequency). Maybe best to just get Teensy 4.1 and save that old Teensy 3.5 for another less demanding project?