You're pretty close! I used INK and a unity INK module to write out the script and imported it to unity. I have a unity script that uses the INK module to read that INK script and extract the narrative lines one by one. I used https://15.ai/ to generate the audio files.
Because I have the audio files separated line by line, the text is actually "called" by the audio file, so there is no need to time it right. When the audio file finishes, if there are more lines of script, it calls the next audio file, which then calls the next text line.