Lore-wise, some of the magic siphoned from the level of the pyramid directly below you comes from the level below them I’d say. Mechanics-wise, it’s on purpose up to interpretation but I’m leaning no, you’re just tracking people you personally recruited so that the number is easier to track and so working your way up seems somewhat grueling and hard to maintain.