Oh wow! I didn't know Godot's web export was this great! Gotta give it a go in the next jam :D
Yeah, all you need to do is use html export and name your file "index.html", then zip all the stuff it gives you as output together.