Driving the pins over 3.6 and under -0.3 isn't great. Technically that's out of spec. But the specs are very conservative.
The clamping diodes should be "small signal" type diodes. 1N4148 and 1N914 are the most common. These types of diodes have very low capacitance when reverse biased. They (usually) have very low leakage current too. All diodes suffer from "reverse recovery time", and again this effect is minimized with small signal diodes. For clamping signals, fast reverse recovery is the most important spec. Typically signal diodes have fairly low current rating, like 100 mA.
1N4004 is what's called a "rectifier diode". They're designed for the lowest cost to turn 50 or 60 Hz AC into pulsing DC current. These are not good for clamping signals. They have higher capacitance and leakage current, which tends to corrupt the signal. The also have slow reverse recovery, which makes the fairly poor for clamping (but still far better than nothing).
Two other types of diodes are commonly sold.
"Fast rectifier" diodes are similar to 1N4004 but with better reverse recovery and usually lower capacitance. These aren't as good as small signal diodes, but they're a pretty decent 2nd choice. These diodes are main intended to be used as rectifiers inside switching power supplies, where the AC frequency is dozens or hundreds of kHz.
Schottky diodes have a very different mix of specs. They have much, much more reverse leakage current than any other type of diode. So a Schottky with a high current spec (like 1 amp or more, such as 1N5817) is usually a terrible choice for clamping signals. But much smaller ones like BAT54 can work in some cases. If you ignore the leakage, everything else about Schottky diodes looks wonderful. They have very fast reverse recovery, and the forward voltage is less. Many people, myself included, have tried using Schottky for clamping small signals because all the other specs looked so good, but the reverse leakage can ruin everything. The reverse leakage is also highly temperature sensitive, so what's not great at room temperature can become far, far worse when you put the whole project into a small sealed box and it heats up slightly.