I found a bug where lil hunter spawned after the portal so I could leave without even fighting him. I am using update 9927 and have the buffs mod and skills mod enabled.
Also this might be fixed, I haven't checked. In a previous update I couldn't leave Y.V's mansion because the portal didn't spawn. I was playing online and I don't remember what mods I had. I do remember I had 9921.