I've noticed in the previous version that the game slows down if the following three things are true:
(1) A city gets too large (in my case, 356 people)
(2) Optimized Tile Drawing is disabled.
(3) You mouse over a business in the big city.
I have not seen as problem with general empire growth (my current game has 683 people over 10 settlements; then again, AI incompetence means three ghoul armies are giving the AI trouble).
Have you tried a game on the 50x50 map? A small map with fewer AIs may help.