Some gems provide the same stat at every level, such as the green gem (20% crit chance). Others scale based on their level, like the blue gem (X health points). If you already have four gems with crit chance, you can shatter the highest-level one for more shards. When you have an excess of blue gems, you should sell the lower-level ones so you can keep the highest-value gem for the blue socketed weapons.
Later on, I will refactor this system, and all gems will scale based on their level. It seems to be a better approach.