Early on I was commiting with git the whole package, that's why I think the git folder just stays large.
At the end I cut the unneeded parts, but the history stays large then.
It definitely surpised me at the end that the project was so large.
Next time I'll migrate only the stuff I need from the beginning.
Thanks for playing and the tips!