Pretty interesting idea! It took a while for me to get everything fully, but I can see a nice little loop now .. I would definitely recommend a lot more visual feedback and affordances, it will help players feel more comfortable. It is nice that there is not a lot of context switching! Great effort, excited to see how this develops!
Would love your feedback on my submission, I think you will enjoy it!