Hi Jean Marc ,
I have check your VGA driver but i have to admit ,the way it's done, it's too complex for me
I have then test on STM32 for try to shift the screen to right , i don't have reproduce the same as we have on the teensy but it come from the H back porch + H Sync + H Front porch , when i set them to wrong value the picture move too
If it's possible , can you try this on your timing ?
Click image for larger version. 

Name:	vga.jpg 
Views:	61 
Size:	55.4 KB 
ID:	22429

This timing made me a perfect 640 x 480 screen with no shifting , now for the 640 x 400 the horizontal timing stay the same , only the vertical one change because they are less line to show.