Okay, leave it with me. This is the first time I've come across this issue and nobody else has reported something like this, so it might take a while and I'm not sure if I can fix it - it could be something wrong with the game or something wrong with your hardware.
If you haven't already I would try disconnecting the controller when you use keyboard to see if there is any weird input signal from your controller that is overriding things. Of course please also make sure your drivers are up to date for both controller and keyboard.