Yeah, I think in the post jam version I'll disable being able to destroy rocks and just out destroying and moving on left click. It may be more intuitive then.
As for dragging not working what were you experiencing there? I hadn't found any issue with dragging rocks. I do know sometimes people will out the rock in what they think is blocking the area but actually miss a bit and there's no clear way to tell other than the guy is still going there. Is that what you're talking about or did you have another issue with the rocks that I never caught?