yup, definitely something I had in mind, and I actually removed collisions with the player colliders while grabbing an object during one of the iterations of development, but at some point changes to collision layers reintroduced the ability to collide with grabbed objects and it never got fixed.
I'm glad it made for some fun trying to break the game and I definitely used the glitch as a fun way to try and float around and levitate my way through the harder parts of the game!