Oh I'm stupid, sorry, I only built for Windows, you have to select the platform to build for. I've built one for Mac and Linux now (they're all the same build version, no cheating here):
For future reference, when you select the “Stand Alone” option in Unity, you still need to build it 3 times (once for each platform). By default, you only have the option to build for your computers platform and you will need to use Unity Hub to install the build support for Linux and Mac.
