The way i would usually get a exact direction, is by centering the sound you want to focus on. And other effects, like a source beeing behind has different effects on it. I think the biggest issue is probably, that if you don’t know how it’s supposed to sound, you’re not aware of what to focus on yet. But you are very right in the fact, that sound localization is quite ambiguous by nature. Especially distance is incredibly hard to get, because sound (volume) doesn’t scale linearly with distance. Also i don’t think i got that quite right yet, it’s quite a challenge.
I’ve been thinking, maybe i should record a playthrough of myself going through this, maybe it’ll make more sense for people then.