How to modify (add filters to) the camera stream that WebRTC is sending to other peers/server

僤鯓⒐⒋嵵緔 提交于 2019-12-03 17:26:07

I found a way out. So basically you need to build your own WebRTC pod and then you can add a hook for using a custom AVCaptureVideoDataOutputSampleBufferDelegate on the videoOutput object. Then handle sampleBuffer, modify the buffer and then pass on to webrtc.

Implementation

Open the file webrtc/sdk/objc/Frameworks/Classes/RTCAVFoundationVideoCapturerInternal.mm

and on the line:

[videoDataOutput setSampleBufferDelegate:self queue:self.frameQueue];

use a custom delegate instead of self.

In that delegate

  class YourDelegate : AVCaptureVideoDataOutputSampleBufferDelegate {
 func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {
        let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)

        //modify the pixelBuffer
        //get the modifiedSampleBuffer from modified pixelBuffer
        DispatchQueue.main.async {
            //show the modified buffer to the user
        }

        //To pass the modified buffer to webrtc (warning: [this is objc code]):
        //(_capturer object is found in RTCAVFoundationVideoCapturerInternal.mm)
        _capturer->CaptureSampleBuffer(modifiedSampleBuffer, _rotation);

    }
}
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!