That's exactly it. The little rectangles in the chimney face in 4 different directions. And you have 4 windows to press, facing the same 4 directions. Just follow the order :)
This is one of those moments where you slap your forehead and sigh.
Thanks for the help btw