Thank you for trying it out and for your review! I'm happy that you see potential in the idea :)
I agree 100% with all your points: this game needs a lot more than what was made for the jam in order to be more playable and fun. I like your suggestions of having a timer or score element - that would make for a good party game mode.
Whichever direction we take, we'll definitely look into expanding on this concept as potentially being our second game.