I figured that installing it would be enough. The installer would associate itself as the default application for .ogg files and that would be enough to fix the problem. So I guess it didn't work?
(To be specific, I believe Fleshcult is blowing up because it can't find this registry key)
HKEY_CLASSES_ROOT\.ogg\Content Type = audio/ogg