Does the clock speed affect the frequency of the OctoWS2811 signal?
Directly, no. The triggering of DMA channels which controls the signal timing comes from a hardware timer with consistent speed independent of the CPU.
Indirectly, you could see a slight secondary effect. The DMA controller runs from the same clock as the CPU. But if it responds to the hardware timer event at a slower speed, you would still expect to see the same waveform output when all 3 event responses take consistent slower time from the hardware timing output to the DMA engine to altering the GPIO control of the pin.
The caveat that matters is consistent number of CPU cycles. It's not always consistent. At any CPU speed, OctoWS2811 has a small timing variation from the ideal WS2812 waveform because the DMA engine sometimes has to wait. The amount the waveform varies from perfect depends on those wait times. Suppose the CPU happens to be reading or writing RAM2 at the exact moment the DMA engine also wants to access RAM2. Since RAM2 is cached, access is always 32 byte cache rows. The bus is 64 bits wide but runs at 1/4 the CPU speed, so if you're unlucky the DMA engine might have to wait 16 cycles for access to the bus to read from RAM2. At 600 MHz those 16 cycles will cause the waveform to distort by 27ns. At 528 MHz the waveform distorts by 30ns. It is an unavoidable problem at any CPU speed, but the slower the CPU speed the larger the effect. Fortunately the effect of 30ns is small compared to the 300ns and 750ns pulses OctoWS2811 transmits. So instead of 9% timing errors at 600 MHz you would get about 10% timing errors at 528 MHz. Typically WS2812 LEDs can tolerate about 50% errors in timing.
The other potential problem is DMA controller conflicts. These conflicts are between libraries which use the same general purpose DMA controller as OctoWS2811. But the DMA controllers for USB, SD card and Ethernet are separate. Those can only cause minor conflicts at the bus access level (same as the CPU), but they don't cause longer conflicts by tying up the same DMA channels OctoWS2811 uses. Audio and some of the display libraries are the main uses of the general purpose DMA. Those can cause much longer conflicts if they are configured to transfer a lot of data in their "minor loop". They can disrupt OctoWS2811, at any speed. But like the bus access, the effect scales with clock speed. These conflicts have been discussed on other threads, and we're planning to support a DMA preemption feature which (maybe) can allow 1 library to have priority over another to solve some of these conflicts. If you have those conflicts, they will probably be just as disruptive at either CPU speed.
But the simple answer is the OctoWS2811 waveform timing comes from a hardware timer that's indepent of CPU speed. There is minor timing error at all CPU speeds, which becomes only slightly worse (not worth worrying about) going fom 600 to 528 MHz.
I think ground bounce is creating a very rare signal glitch, so I'll either add more power connections or experiment with the timing in the library.
