Or maybe it's another problem?
I tried turning off the portraits and now the game is fast again
Yes, avatars load a lot of graphics into memory and that's why lag increases.