Wireless Xbox One gamepads on mac with Unity is known to have a different key mapping – from my experience you can try setting a different mapping specific for this configuration or use the the Unity's new input system for gamepads as it fixed the issue.
The v0.9 of the new input system does have some issues with keyboard on some Windows machines though...