How to access audio result from Speech Synthesis API?

Question

The Speech Synthesis API allows text-to-speech functionality in Chrome Beta. However, results from TTS requests are automatically played by the browser. How do I access the audio results for post-processing and disable the default behavior of the API?

Answer 1

There is no standard audio output for the TTS system and that seems quite intentional so it is unlikely to change anytime soon.

To understand why, you can look at the other side of this interface where a browser extension can act as a TTS Engine and provide the voices the client can use:

Being a valid TTS Engine accessible by this API in chrome is about supporting starting/pausing/canceling and resuming of TTS requests and sending updates on the progress as events of the following types:

https://developer.chrome.com/extensions/tts#type-TtsEvent

As such, there is no standard way for a TTS engine to indicate the resulting audio aside from actually playing it. Depending on the specific TTS engine, it may not use a standard audio format or even the browser's normal audio devices access. (For example, it may be forwarding the text to the platform's accessibility system.)

If you know something about a specific TTS Engine (or create your own) then you can build your own interface¹ to retrieve the audio file. But that TTS Engine must then be installed on every client's browser where you want to use it. This is why any solution must point you to a specific TTS Engine or an outside TTS solution if you want to control the playback beyond adjusting valid inputs to a TTS Engine request (relative pitch, relative volume, relative rate, sex.)

Notes-

¹ If you give a TTS Engine such an interface, it can not trivially extend the existing TTS event API since the browser is checking them:

// attempt to add properties to an otherwise legal event in an Engine:
sendTTSev({'type': 'end', 'charIndex': len, foo:'george'});
...
Uncaught Error: Invalid value for argument 2. Property 'foo': Unexpected property.
    at validate (extensions::schemaUtils:34:13)
    at Object.normalizeArgumentsAndValidate  (extensions::schemaUtils:117:3)
    at Object.<anonymous> (extensions::binding:361:30)
    at sendTtsEvent (extensions::ttsEngine:17:22)