A fun idea for gameplay, but as others mentioned, some kind of in-game introduction would be really useful because it can take some time to figure out what to do. After that point, it's pretty enjoyable, although not easy at all. Maybe starting out at a lower speed (both for the player and the NPCs) then increasing it over time could adjust the difficulty curve for the better.
One other thing: in some cases, when I stopped, an NPC also stopped at the same spot and it was kind of hard to see if they are still there or not.