Ok, so to use it you need to install it first using "pip install auto-py-to-exe" just like you do it with pygame. Next you run "python -m auto_py_to_exe" command and the window appears. In that window you select the path to your .py file and hit convert .py to .exe button on the bottom. And it should work. I think that your assets would need to be near the .exe file just like they are with your .py file.
If it does show errors, google it how to convert pygame project to exe using auto-py-to-exe.
I never tried pygame, I use other python module named ursina engine, which needed some additional steps in auto-py-to-exe to run from executable propertly. I think it will work just fine with pygame. If you need more inforomation, here is the link for the documentation: "https://pypi.org/project/auto-py-to-exe/2.5.0/".
So try it out and say if it works for you.
I wish you luck with this and have a great day.