I really like the current system its very unique. The issue is more that the ai is really buggy and just does not pick up the stuff. But I do like there being basically 4 types of villagers, those that extract the resources but don't pick them up, like miners, those that pick them up (which u then divided by those who just bring everything to storage and those who bring it to current building process) and soldier. I can see that the soldiers take a lot of inspiration from Kingdoms, I would try going for adding a melee, like swords to it. Overall, its very cool i would stick to your initial ideas, the issue is more that it has bugs, waves of enemies are very unbalanced and that's it.
I agree with you, that's why i've given multiple ways to fix the problem. Clicking to pick up was just one of the options and it'd be a hard skill to achieve in the skill tree. Also i like the melee idea aswell as nerfing the enemy waves a bit, maybe adding more types of enemies instead of just adding +1 to each wave