The lost house key will lead to the last shrine, and it is inside the decorative vase. It should prompt you to take the key if you examine the vase and look inside it. If it did not work, let me know. I have a patch coming through for the bookshelf issue, it has to do with loading the game.
I'm glad that you had fun at least! Thanks for playing!