I like the base idea here - one of the things that I really liked was when their die hit mine and it restarted my counter. That adds a little bit of strategy into where I place the throws.
I think a good evolution to this would be using telegraphing to the player (like making it a different colour after it lands and starts winding up to shoot) what the next dice to shoot would be, and I can try and aim a die at it to interrupt it. That feels cool! At the moment it just feels a little rolling dice at someone and not really playing the game.
Still, the physics is satisfying and there is a great core idea here! I think you should evolve on it