Cool idea, but the way it is implemented it does end up feeling like you move the red square, even though you are moving the level ;)
Maybe add some mechanics that are only possible if you move the level? Like the 'player' ending up outside the screen sometimes, or something like that?