Interesting concept!
In the current version it's a bit simple (normal for a 3 hours jam) but if you had animations happen regularly while you walk and react on how well you are doing, etc... now we're talking. And that sounds like what you planned from reading your description :)