making puzzles is hard and making them for some concepts will make you doubt the concept, unless it's outright goldmine of exploration even for the designer, kind of like Braid. but I disagree, this core idea is properly worthy of a full game&release. But yes, very heavily depends on good puzzles, the gif seems like a good maybe second or third one.
But you need to quickly introduce some other physics elements see-saws, pulleys, levers... only then the mechanics will start to properly unravel and be interesting, you have one "non-standard" gravity thing (both of the players), now you have have to add some standard gravity things that people instinctively understand, and start playing around with how those two cathegories interact (possibly in interesting ways) ;)