These things happen in gamejams, don't sweat it! FWIW I think its a good mechanic, I wouldn't remove it just fiddle with the sensitivity.