Uncompressed, unencrypted, unaltered, raw transfer of real-time PCM audio data through WebRTC stream

有些话、适合烂在心里 提交于 2019-12-02 13:11:41

问题


I'm transferring a live audio stream between 2 Electron window processes using WebRTC. There are no ICE or STUN servers, or anything like that, the connection is established manually through Electron IPC communication (based on this code).

Note: from the technical point of view regarding the audio streams themselves, this is very similar (if not identical) to streaming between 2 browser tabs on the same domain, so this is primarily not a question regarding Electron itself, although Electron IPC would be obviously substituted with a browser equivalent.

The audio stream works, I can transmit audio from one window to another in real-time, as it is generated. That is, I can generate audio (Web Audio API) in window "A" and listen to it through an <audio> element in window "B", or do processing on it using a separate AudioContext in window "B" (although there is some latency).

However, the audio data is vastly altered during streaming: it became mono, its quality dropped, and there is significant latency. After fiddling around I've learned WebRTC does pretty much everything I don't need, including encoding the audio stream with an audio codec, encrypting the transfer, running echo cancellation, and so on.

I don't need these. I need to simply transfer raw audio data through local WebRTC without altering the audio in any way. It needs to be float32 accurate to the sample.

How can I do this with WebRTC?


Why use WebRTC then?

I need to do custom audio processing inside the Web Audio API.

The only way to do this is using a ScriptProcessorNode, which is unusuable in production code when there's essentially anything on the page, because it is broken by design (it processes audio on the UI thread, and causes audio glitching by even slight UI interactions).

So basically, because of this (and to the best of my knowledge), my only option is to transfer audio with WebRTC streams to another window process, perform ScriptProcessorNode processing there (nothing more is happening in that window, empty DOM, so the processing is always nice and smooth), then send the results back.

This works, but the audio is altered during streaming, which I want to avoid (see above).


Why not use AudioWorklet?

Because Electron is 5 versions behind Chrome unfortunately (version 59 at the moment), and simply does not ship AudioWorklet yet.

来源:https://stackoverflow.com/questions/48791065/uncompressed-unencrypted-unaltered-raw-transfer-of-real-time-pcm-audio-data-t

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!