I am not exactly sure what would be the best way to implement it but something related to the collisions or creating an invisible sphere around objects attached to them are probably some viable ways to fix the issue. Thanks for the experience!
Thanks! Did you manage to finish the game despite the controls?
