I'm working for a few days on a MultiFX box with a Teensy 4 + Audio shield.
For the FX controls I based my work on the schematics available on Teensy site :

Click image for larger version. 

Name:	schematic_ioexpand.png 
Views:	10 
Size:	10.2 KB 
ID:	18296

My config is quite similar, i got 3 74HC4051, 1 linked to a digital pin (9) for arcade buttons and 2 linked to analog pins (16,17) for pots.
Arcade buttons have a led in it that I connected to digital pins (4,5,14,15) to turn on and off.

Here is the panel layout of the prototype box :

Click image for larger version. 

Name:	IMG_20191130_133100.jpg 
Views:	9 
Size:	253.1 KB 
ID:	18298

Everything is working quite well except that my pots are jumping the center values. All pots are jumping from ~500 to ~650 when I try to center them.
I'm using 100k linear pots.

I measured the voltage coming out of the pots with a multimeter and I read values from 0 to 3.3v with apparently no jump on center as seen once I read values with the teensy.
I'm using ResponsiveAnalogRead to smooth the values but I also tried to check raw values and I see this jump too.

I think I saw a similar problem on the forum, advicing to put capacitors on the circuit but that was not really clear to me.

Could someone give me some advices about this issue ?

Thanks for your help.