Hi Phariax,
Most of the pipeline is automated through the Projects/Main/Tools/bake.py script. You'll need to edit that to point at your paths (see BLENDER_DIR, PYTHON_DIR, etc.) and there might be some other headaches—looking at it again, I see this was written for Python 2 and there's a few things to change if you use Python 3 (print 'foo' becomes print('foo'), mainly). It also expects Cygwin to be installed, for the cp and mv commands.
If you can get that script working, then you should be able to drop new .blend files into Projects/Main/Raw/Meshes, and then bake.py will invoke Blender on the command line and export the mesh to /Intermediate/Meshes (using that xmlmeshexport.py exporter). Then it will run MeshCompiler to convert that into the custom .cms format.
None of this is documented, because the engine was never really meant for anyone but me. But I'm happy to help if you've got any other questions!
David
Edit: To answer the question directly, the .mesh file is the XML file that MeshCompiler expects.