That looks more like the packets are out of order does it not? When just viewing the raw audio file you shouldn't have this specific issue even if the clocks are different.