On the second day the Mansion Upgrade becomes available; it helps if your slaves have some magic affinity.
Thank you, I actually read about the upgrade minutes after I've posted this comment :D Looked weird to me that the mod mentioned the need for the specific upgrade only once in the segment that's pretty easy to skip accidently.
