Yes, or maybe, depending on your idea of simple, and no.
Yes, you can switch by just writing to SIM_CLKDIV1. But not all combinations are legal. The hardware has some smart logic to detect and ignore writes with illegal combinations, which you could imagine as making thing simpler (at least horribly wrong things don't happen), but I can assure you it feels pretty frustrating when you're experimenting and all your mistakes result in no apparent changes because there's no feedback you wrote a wrong combination, other than everything stays as it was. However, the logic doesn't keep you from overclocking, and trying to overclock the flash too much will result in crashing.
And no, your assumption about not affecting the peripherals isn't quite right. Almost all the peripherals run from F_BUS, which is also controlled by SIM_CLKDIV1. If you alter F_BUS, all the peripherals change speed. If you go with F_BUS at only 24 MHz, then you'll have much more flexibility to alter the CPU speed. If you use the faster speeds which are the default, those rules about which combinations are legal will come into play and give you very few choices that don't mess up all the peripherals.