Ah I see, there are two full screen buttons (a blue, twin arrow one, and a black, x arrow one), I guess I had to click both at the same time or something for it to work.
