I think I found a bug, when I equip my slaves with shackles to avoid them running away while training them, the movement restriction stays there even after I unequip them. It doesn't seem to change no matter what :(
Let me know if I can provide the save for reference.