Because they keep following were the mouse is. So they kept running faaar away from your village :p
if u really want to pause (as i have) you can put your mouse in the middle, and then click the windows icon(on windows) or the command icon(on mac) to exit
Hope this helped!
