Okay, it seems as though there are some bugs with Unity's Xbox controller support. Did you perchance have two or more controllers attached to your PC when playing?
Viewing post in Development build 1.0.1d1 has been released!
This *should* be fixed in the latest build: https://8bitape.itch.io/swords...
Essentially it only listens to input from one connected device. Would massively appreciate it if you were able to give it a spin and let me know how you get on!