I know why it does that, because I've had that problem before. Either you have the canvas that the button is attached, to you either set it to Screen space overlay, or if you set the canvas to follow the camera you didn't set the canvas to scale with screen size, meaning that when the game doesn't go fullscreen the button is still in the spot it was when the game is fullscreen, which isn't in the frame. I hope this helps!