are you getting the hardware warning? some ren'py games act weird when not launched in fullscreen, such as not using hardware acceleration.
it does not give me a fullscreen button but it replaces it with a "float" button