After some testing I figured out it only happens when I connect the controller after opening the game. So it's kinda my fault. But, it's still technically a bug.