I noticed this issue even on the default Grombit character. There's a substantial (half second or more) delay between when I say things and when it responds. Hell, the HONK and David characters, who are much simpler, still have a noticeable delay.
I hope this is something DreamToaster can figure out and resolve. The program seems fantastic otherwise, but that's a major sticking point for me.
EDIT:
https://itch.io/t/1795229/audio-y-lipsync-bug-solution
This does seem to work, though maybe temporarily? It seemed like things were sometimes still starting to delay after a few minutes.