So for anyone dealing with the living room glitch, I found something of a workaround. So it seems when someone is in the living room, like Jess doing yoga or Ami cleaning, it won't pop you out. This is usually in the afternoon and some mornings(still don't know the exact schedules of all the characters).It's the default, empty living room that's glitching.
EDIT: I'm using the Android version, so I haven't tested this workaround for Mac or PC and can't guarantee it will work there. Though you're free to try it.
That said, accessing the garden is still normally impossible except under special circumstances, and that's where the work around comes in. Very minor spoilers ahead.
To get access to the *note from gramps* in the garden, which is automatically triggered at the start of act 2, you'll need to advance the story along enough until you're told members of the Fey council want to meet you. This scene jumped you directly into the garden where you'll meet a couple council members. At the end of the scene you'll be left in the garden with the *note from gramps*, allowing you to continue progress. If you time when you trigger the council meeting correctly there should be someone in the living room allowing you to pass through it without the game crashing.
Later when you can visit the various demense of council members through the Nexus you'll meet Roxie. Progressing her story will result in you having to return to the garden again to get a flower. Next you'll want to advance the Organization storyline enough and it will eventually have you needing to talk to Chloe. When you knock on her door Pixie will tell you she's in the garden and again you'll be jumped directly there. At the end of the scene you'll be back in the garden and can pick up the flower to continue the story for Roxie. You MUST trigger the event for getting the flower BEFORE progressing the Organization storyline. Otherwise the option to collect the flower won't appear. Again, if you time when you trigger the scene with Chloe in the garden correctly there should be someone in the living room allowing you to pass through without crashing.
Same idea with the first bath with Pixie, though you won't be able to return to the bathroom again afterwards.
Phone calls from Emma in the living room work on the same premise. Time your trips into the living room with when someone is in there and you should receive the calls fine. Keeping multiple saves is highly recommended in order to avoid getting stuck in the garden and unable to leave without crashing.
Unfortunately, can't find any way around the bug in Asteria's quest line to get the tournament going as that's a bug with the UI. So unfortunately unless someone else finds a workaround we're stuck waiting until it gets patched. Hope this helps people at least with progressing al little further until then.