I ended up having more fun throwing bricks at the people on the sidewalk and didn't realize it took away points.
Only if they're Christmas related will they take away points. The actual people/fishman are fair game and give points, so brick them to your hearts content.
