Thank you! :)
It seems like others had the same problem. So it's not on you! The intended way is to talk with the construction worker rabbit behind the bigger rock pile. You can talk with him, when you stand right above him. He will blast them away.