So, the emission color had nothing to do with the problem at all. I found the cause of the problem here, might be worth giving a warning to people converting this to URP:
Just turn off "Enable Mesh GPU Instancing" (it's turned on in the prefab) in the Renderer module of the Particle System.
White emission color doesn't work btw, it would just change the entire thing to white, the colors won't show anymore.
I found black to be the correct emission color to use.
