For a charging, I would use dedicated circuit. Like "Li-po rider" from seeedstudio.com. It should handle charging and switching. And supplies steady voltage on output, so you don't have to worry about anything. The one I mentioned gives 5V, so you can just connect it to Vin.