In response to Ericdef's comment. In Unity you can put grabbables and the player rig on different physics layers to prevent prop flying or other manipulation of the player with props. Just make sure your hands are on a layer that actually can interact with the grabbables.