For some reason, the entire screen moves instead of just the character.
That's intended. You're moving the screen and the dwarves you see are just NPCs. I will be adding an actual player character in the final build though— it's probably going to make it easier to interact with things.