That's because gem upgraded to lrvl 1 count like 2 normal gem, then not only double the damage bonus charged to the cards, but also count like 2 gems for skill charging.... for example a level 4 gem count 5 ( base gem + 4 upgrade )