In the project settings you need to enable advanced settings then go to Audio -> General -> Default playback Type.web and make sure its set to stream per the documentation:
https://docs.godotengine.org/en/latest/tutorials/export/exporting_for_web.html
To use Godot's own audio playback system on the web platform, you can change the default playback mode using the Audio > General > Default Playback Type.web project setting, or change the Playback Type property to Stream on an AudioStreamPlayer, AudioStreamPlayer2D or AudioStreamPlayer3D node. This leads to increased latency (especially when thread support is disabled), but it allows the full suite of Godot's audio features to work.