Interesting concept. Tough to navigate without more audio clues (which I know is the whole point). But by making the audio the focus, I think you did exactly what you set out to do.
Extra kudos for experimenting with making games more accessible. Good job.