So just a bug then. RIP
I think going to the park with Anna keeps Josef from asking, so you have to choose between hiking and the park.
It's a bug, I just have to reload the save on Wednesdays and Josef will ask to go. If I don't reload the save and go Upstairs he just won't be there and won't ask.