The pathfinding system is a bit complex, but in essence it calculate the distance from the edge of one key to the edge of another key to determine if there is a valid connection between two keys. The margin is quite large so it should just work (- Todd Howard).
Do you remember what level it was and with which keys?