We've been working on solving this issue for the past month, but it's next to impossible to pinpoint what exactly is causing it. If you can, could you share your save files? Also, can you specify what platform are you playing on?