This idea is incredibly creative, an actual physics system is a great way to "load the dice"!
The two main things I think could be improved are the controls (you could either go for more Octodad chaotic controls, or maybe more precise controls and focus on a more puzzle feel) and the "level design" (with more time, in limited supply during a jam, more levels with more specific requirements could be developed). It would also be great if the rolling of the dice were to be sped up (which should be possible with a physics system? I'm not too familiar)
The inclusion of an interactive tutorial was also a great touch! (I think without it, the mechanics could have been a bit confusing.)
Great job on your entry!