Flags are too binary, I was thinking number values (I call them "traits").
How to not flood player with thousands of memories is a real problem. But there is a good real-life mechanics for that: forgetting.You gradually forget about things, starting with smaller and more mundane ones. Also, some (most) of memories can be not available immediately, but you can be reminded about them by something related. Such as, an item in your inventory. For example, you can look at a sword and recall how did you get it, who you fought it with, with who you were talking about it etc.