It's actually only a problem when walking back. As I wrote earlier, the setAttribute('tabindex',0) thingy fixed a similar issue, and it took me days to figure it out, so I thought I might save you that waste of time. However, my issue was my game wouldn't get any keys input at all, despite clicking the canvas, but the input was somehow stuck to an event handler of the surrounding page.