Yeah, it does happen sometimes. However, you could use it to your advantage, so you don't have to worry about fake objects going into the real hole. Thanks for the feedback!
That's what I was thinking at first but then I realised I got points when they went in, so I had missed out on a lot. Still enjoyed it though.