I think it happens with overlapping objects mostly, and I'm not sure about the best way to disambiguate. Maybe having a selection state and an outline around the selected object would help?
Either way it's quite a minor nitpick on an otherwise great game.