I made the whole house a collider to be shot on, so normally it should work on the whole house.
Did you shoot on the houses with the flags on them?
