I think a better solution would be add an X Ray shader that lets the camera clip through the walls but still show the player through the buildings(something like this ).
That's actually the perfect solution, thank you so much!