Google Cloud Text-to-speech word timestamps

Question

I\'m generating speech through Google Cloud\'s text-to-speech API and I\'d like to highlight words as they are spoken.

Is there a way of getting timestamps for spoken wo

Answer 1

This question seems to have gotten quite popular so I thought I'd share what I ended up doing. This method will probably only work with English or similar languages.

I first split text on any punctuation that causes a break in speaking. Each "sentence" is converted to speech separately. The resulting audio files have a seemingly random amount of silence at the end which needs to be removed before joining them, this can be done with the FFmpeg silencedetect filter. You can then join the audio files with an appropriate gap. Approximate word timestamps can be linearly interpolated within the sentences.