The idea is interesting but in practice it's incredibly frustrating as you spend most of the time waiting for the right controls to appear. If you plan to develop this further I'd recommend adding icons to show what controls are queued up next and perhaps some multiple paths so that you can adapt your route on the fly.