Hmmm...you make an interesting point. Maybe doing a limited edge pan would solve this problem of "Enemies can see you from off screen."
