You can't reduce his popularity around masses in place with no access to masses. Popularity only increases and decreases as a result of choices in quests of Verdaux, not any other way. Maybe someone will give a hint on how to beat him with such popularity