I noticed that too. After about 30-40s the issue seemed to go away, at least on my computer. I'll definitely need to investigate. In your case does the stuttering continue after an extended period?
Not really, only when I look into a corner so the framerate rises. Seems to be directly linked to the framerate, audio buffer update on frame event. You maybe can increase the audio buffer size (That was unity ?), or if you want to go through the troubles of platform-dependent branching, in WebGL you can always use HTML5 sounds directly. For instance myaudio.play() starts a sound, once it's loaded, and restarts it if it ended, but is ignored when it's already playing. There's zero buffer complications, as the sound is managed by the browser directly.