We tried it again with a mouse - turns out the issue was that we didn't quite understand the mechanics and that you needed to switch slot with E. That could have been clearer in the instructions.
thank you for the second chance! Turns out that guiding players and tutorials are harder than I expect