This is strange...
Is it an xbox360 or xboxOne controller? Which OS do you use?
We only tested the game with these two, and we are not sure if any other controllers would work properly, sorry.
im on windows 8.1 using a switch pro controller so thats probably why now that i think about it