However, the problem is that he progresses way too slowly,
I reiterate: it takes the MC 6 months tops to reach the stage others take years and often decades to achieve. Starting effectively from mortal level scratch. If this is your idea of "too slowly" then we simply have to agree to disagree on this.
Regarding the stones, Tang Wei needed a few of those to replenish his qi reserves and teleport out of the game world. But he's already a fully developed god-like being, not a fresh vessel which needs incomparably more qi to develop to his level. Also keep on mind that what your MC gets their hands on aren't the same stones, but tiny remnants of stop-gap solution produced by Tang Wei out of "mere" few millions of the world inhabitants. They're far from the same thing in terms of carried power.
Btw, each of the stones you get in the story grants you the same amount of power: 10 qi and 100 exp. The rest of exp you have available after the time skip doesn't come from the stone, but it's what you've accumulated as your character attributes up to that point. Perhaps it's a narrative mistake to make the first instance of getting your hands on one of these pebbles seem far more beneficial than it actually is.