I like the premise of this game! The annoyance meter adds a nice bit of tension to doing the tasks and fits well with the game jam theme. The task system is solid and a good base to build on if you choose to develop the game further. The way the villagers wandered around the village was a nice touch.
That said, there is definitely room for improvement. Others have already pointed out the strange way the houses are placed, but I also noticed that the trees have a much smaller collision box than their sprites. Did you mean to set up Y-sort on the tree sprites? That would prevent the player sprite from rendering in front of the leaves. I also encountered a bug where object interaction text appears even if the objects aren't visible or interactable:
Overall, I liked the game. Good job on the submission!