I would check the input direction and then according to the current cube normal i would use different anchors/axis for the rotation, when no cube and valid move id use 180 degrees as opposed to 90. (im hoping to go over it in a video on my youtube channel zerokelvintutorials) As for moving the cube through the illusion that was a bit trickier, having to toggle on/off different layers. Seeing as how you programmed similar mechanics, how did you keep track of what number of the die is "up"? I use raycast towards -normal to see what plane it his (my cube is made of 6 faces) but I wonder in what other way it could have been solved.
Wow! That's really interesting, I had a feeling it was something to do with toggling layers but I couldn't put it all together myself. Really impressive. For my game, I used Unity and kept track of the object rotation and original face direction vector. Each time you move, it updates 6 vector variables based on the new rotation, where each vector corresponds to a side. To check to see if a certain side is face up, I simply just check to see if the face vector is equal to Vector3.up, or (0, 1, 0). I tried to keep it as simple as possible and this solution works pretty well. I hope that makes sense haha.