Hi, definitely not retarded. Other people met the same road-block. While the first growth stages are only tied to strength-gain, Lvl. 3 and 4 require some progress in the main-story line as well (just need some story elements presented before my girls get too big, otherwise it won´t make sense later). There´s a little button with requirements in the top-right corner, but for Lvl 3: Check in on Stella at the brothel a few more times.