Hey thanks for the report! I put that bug in my bug list. You probably noticed the rebinding menu doesn't quite look or feel like the rest of the game. It's provided by Rewired, so it always behaved a little weird. But I think we should be able to do something with this issue, especially just preventing the main menu in the background from closing when the controller changing menu is up.
Thanks! -Andy