That is actually what's happening behind the scenes to make things easier when putting it together! The player scale and camera zoom are just synced so that it looks like the world is what's scaling instead.
We could definitely do with a "recentre" button or something though so you can keep scaling if the window gets too big or too small.