In an attempt to get windowed mode working, I used Universal Unreal Unlocker to enable the in-game console, and it seems like it's constantly setting the max fps each frame, and since it immediately goes back to fullscreen if I try to make it windowed mode via the fullscreen command toggle it might be constantly setting it for fullscreen as well? I'm not sure that this should be happening, I've never seen this in an unreal engine game before.