Did a Goblin run up to you just before you first visited the house?
EDIT: There was a bug related to rainy weather that broke that step of the game, I fixed it now so it should work, but you will have to restart the game for it to work though :(
Thanks for the heads up btw :)