After playing around with it more, I've realised that the neutral-only sprites are surprisingly convenient sometimes. It feels much easier to manipulate the scene without the random emotions throwing curveballs. I imagine having neutral-only sprites in addition to emotional sprites would be rather difficult to work with though.