We decided to reduce the collider on the player which has made the game far less clunky. It no longer feels like everything is a giant square!
Thank you for the feedback!