There is a small hidden area near the abandoned laboratory where the jewel can be found. But a certain ability is needed to remove the boulder that is blocking the way to the hidden entrance.
Viewing post in Feedback (Early Access Version)
Currently there are five world fragments, each in their own dungeon with an accompanying quest. As well as four world shards scattered across the world.
Could be a bug if one wasn't counted if you did four of the world fragment quests already. Luckely they currently serve no purpose so nothing can be missed.
You also can check finished main quests. The world fragment quests should all also list a world fragment as quest reward once the quests are completed. Or check the world fragment number on the save file screen, if it is the same as the amount of world fragments in your inventory.
Would be two easy ways to check if you want to know for sure.
Thanks^^ Actually was one of the first times drawing dragons.