Fraps, which captures OpenGL (or equivalent?) directly. Perhaps the game has a frame and only in the center it was OpenGL context, so that it was displaced by Fraps, which for some reason still used the resolution of the whole window.
Would have used OBS if I had known. :)