Its an interesting idea, I actually tried a similar type of game for a previous jam. I like the core gameplay loop, however it doesn't have much progression, which discourages the player from really caring.