So I've been able to reproduce your issue to an extent with wireless Xbox Series X controllers, where one controller controls two characters (I don't know if this is the exact same issue you're having). While I am still trying to fix the issue, I did find some remedies:
1. Playing with the controllers wired seems to help
2. Focus on another window while keeping the YMC window in view gets rid of the double inputs (this one's weird)