Hey! Are you playing on PC? Because if you are, right click will take you back to the game from any type of menu
I know about this issue with lookin up memories from the journal, but despite it appearing like a simple bug, it requer a lot of quite complex code rearrangement. So I figured it might look a bit ugly, but as long as it works, it works.
Guess I should add fixing it into the shedule after all