Have you brought up task manager and checked memory usage? My bodies and portraits folders total 850MB, I can load up a save with 37 residents, and the game is using only 210MB of memory. Can do some actions, some combat, etc. and memory usage doesn't climb much, 263MB.
I can even load the Portrait menu with delayed portrait loading mod, and memory usage only climbs to 296MB. It'll go higher if i scroll around a bunch but not too rapidly. If I load up the bodies folder, it actually clears what was loaded from portraits and drops down a bit before climbing some, but still around 300mb. I do see the commit size getting really high, like 3GB, but it's not actually using that memory but you might still hold up things, or cause an issue if you don't have much free, especially if you have your paging file disabled.