I see. I noticed that the "playing field", the slider interface, and the tavern guy's interface each keep up differently with resizing.
For some reason, the directional controls for the slider and tavern guy remain bound to zqsd and do not acknowledge modified bindings (that's why "keyboard only" seemed to do nothing). The action button acknowledges both default and modified bindings. Outside of these contexts, only current bindings are acknowledged.
This kind of context-triggered "rebind denial" only happens to me in recent games, though I'm not sure if only in Unity games. I work around this using AutoHotkey.