You don't lose progress for not going to the gym, you lose it because you don't sleep. You should sleep 8 hours everyday to not get sleep deprivation effect which drains you of XP every hour.
