So I tested the demo and it works fine, and when I set mine to the Z axis, it also works fine. Both both X and Y have this problem. Is there another way I should be rotating them?
Viewing post in Cube squishes during rotation
Interesting! I remember testing the x / y rotations and they worked initially, but that was v1, so maybe I've got some code weird in there. IDK why it would mess with it though since essentially all I am doing is setting a matrix for the rotations. Hmm... But yeah, sorry I can't help more! I will have to look into it when I have time. Fauxton is mostly un-supported for now (I've been very busy with other projects). It's best used as a starting block that you can then expand from. When I get some time I will try to look into it and see if I can pinpoint the issue. X/Y rotations with sprite stacking are less than optimal due to the fact you will see 'gaps' when rotated at certain angles. I highly recommend Stack3D as an alternative if you cannot get it to work, as I know 100% he has solved a lot of the issues I have not been able to. I also highly recommend this series on 3D in gamemaker if you want to learn a better understanding of how 3D in-game maker works. :)