In my head, the rules were as simple as:
- appears to the left = orange
- appears to the right = blue
- Things that you see don't change as long as you can see them.
So the tiles above and under the character (I think that's what you mean by "at the sides") will be of a different color whether you see them from a clockwise or counterclockwise turn.
I will say that understanding the mechanics was part of the puzzle experience :)