(A very) long time ago, on a Rabbit 3000 processor, I removed the FAT layer to increase write speed. No big gains, but it worked. At that time, not much RAM for buffers, no DMA, 8 bits Z80 CPU at 54MHz, ....
And then, I discovered that writing a file in a directory was what caused low speeds...