You can totally play with multiple people! I would up the chance to have multiple enemies for sure. The rules already allow for increasing the number of bad guys as you get deeper into the armory...d2 after the 2nd special room and d3 after the 3rd. You could start at d2 for 2 players and go up to d3 and d4 as you complete rooms. You might want to add a minion enemy to the special rooms as well.
I'm super curious as to how this would go, so please reach out with a play report if you manage it! Thanks!