I found another way to fix this. After you go invisible just go into the temple to the area where you can see all your achievements. The characters will reappear
Edit: This unfortunatly breaks the fertile blessing mechanic., so the first option is probably better