Nice jam. I think a mouse driven gui would be nice, but I really like the concept, its like the modding is built in to the gameplay. Maybe there's a way to make this more seamless, like perhaps only use the mouse for platform building and then only use keyboard/gamepad for movement.
