Eventually you'll be far enough away (past the draw distance) where he know's you can't see him even if you're technically looking in his direction.
