That sounds like it can be frustrating to me. I will make it so the closest key that you can get is seen through walls if you do not find a key within a certain amount of time.
