It definitely makes the game a bit harder, but in my opinion that is bad way to make it harder in this case. It felt to me as a player that game was… cheating. It was hard enough trying to switch the right holes under time pressure in 3D environment. If you want to make it even harder here is one suggestion: make more shapes and with right mouse click hole will change shape. Now player has to act both color and shape. Of course now you have to give more time, but player testing helps to get it right!