Hm, I feel like doing that would pretty drastically change the original idea, however that's not necessarily a bad thing since as you say, this idea doesn't really work that well xD
Definitely interesting suggestions!