Definitely debating this at the moment. On one hand, I worry about too much complexity over time if people aren't removed from the strands of the narrative, so to speak, but I do like the option to revisit them.... Perhaps a neat little optional mechanic?
(Or feature creep is happening right now, as I type!)