Ah! That actually helped! Even though it was windowed, the Maximize button was greyed out. Indeed, the Alt+Enter works, and I can play it fullscreen now. Thank you!
