Game loads all the images in the folder, which can take quite a while, and a lot of memory. If you have a lot of portraits, especially high resolution, which tends to be worse for full body images, it'll take a while to load period. If you also don't have a lot of ram, it can take damn near forever as it hits swap. I've got 847 body images taking up 405MB on disk, loading them takes several seconds and hangs the game for a bit on my decently powerful system. The game's memory usage also goes from 187MB to 1284MB. If you have even more and/or larger portait/body images, then this could grow greatly.
Ideally the game UI would load these in the background, and not load everything, but only the items actually visible and maybe some before/after. That however isn't exactly simple to implement.