If you do play with arm_set_clock() after startup, I'm curious to hear about your experience.
Paul, does the above apply to the Teensy 4.1 as well? I have been using
arm_set_clock() in my main project running on a T4.1 for almost a year, and haven't once experienced/identified any negative effects related to using this, at least that I have noticed, with the processor or any peripherals in use. It is a user changeable setting in my app from 20-816Mhz; I frequently change it manually "on the fly" to see it's impact on power consumption, UI update speed, ADC library read speed, video playback speed, etc. Additionally, I have a user defined setting for "power saving" which after a period of time, turns off the screen and steps to a lower CPU speed; a touch on the TFT returns it back to the "normal" speed, and have never seen any issues related to this during operation.
Peripherals/functions/libraries used in the app include I2S2 audio in/out, SPI (for touch screen and a digital potentiometer control), GPIO6 for 8/16 bit parallel TFT screen writes/read, bit-banged I2C reading an INA219 current sensor, Serial8 for TF02-Pro distance sensor, NativeEthernet, LVGL graphics library v7.x, and intervalTimer amongst others, and all seem to function correctly after calling
arm_set_clock()
General application use makes very few calls to this, however. Mostly switching to the user defined CPU speed read from EEPROM on startup and switching in and out of "low power" mode. However, on startup, my app can programmatically walk through a small loop reducing CPU speed in steps if TFT initialization fails, to retry initialization at a slower CPU speed. This is mostly due to the Teensy operating too quickly for the NT35510 screen controller at higher CPU frequencies and me constantly tweaking the number and locations of NOPs in my driver to squeeze maximum performance out of it, and sometimes getting it wrong...this looping doesn't seem to cause any issues, either.
For me, your dream to support dynamic clock scaling at runtime is actually a reality.