It has something do do with one of the cards, yes.
Yes the key exist since version 0.7, I think one book in the library gives hints on where to find it. And one of the tf hints as well. But yeah that is is very well hidden.
Viewing post in Feedback (Early Access Version)
Currently there are five world fragments, each in their own dungeon with an accompanying quest. As well as four world shards scattered across the world.
Could be a bug if one wasn't counted if you did four of the world fragment quests already. Luckely they currently serve no purpose so nothing can be missed.
You also can check finished main quests. The world fragment quests should all also list a world fragment as quest reward once the quests are completed. Or check the world fragment number on the save file screen, if it is the same as the amount of world fragments in your inventory.
Would be two easy ways to check if you want to know for sure.
Thanks^^ Actually was one of the first times drawing dragons.