Boost bandit's moral and attitude towards you. Easiest and fastest way - go to Jester and give them money. Talk is available 2 times per day (50 coins each). With that they will stop visiting your house.
It work, Thank you!