I think this idea is really good - but honestly a bit wasted in the real time context. Imagine having to roll in turns and assigning the dice to the best fitting character. It would become a really cool puzzle game. I dont think its bad at all how it is right now but i personally like puzzles more than quick math and reaction times.
The different classes are well thought out and could probably make for even a fun board game.