That's what I was thinking. Update isn't great at dealing with physics, you should use fixed update. It's as simple to use but works way better. And I don't know if you did, but late update are great to put on camera, it help reduce the jittering. Hope it will help you in the future. Good luck.