Display getUserMediaStream live video with media stream extensions (MSE)

问题

I am trying to display a MediaStream taken from a webcam using getUserMedia, and to relay it to a remote peer using whatever mechanism possible for it to be played (as an experiment). I am not using webRTC directly as I want control over the raw data.

The issue I encounter is that my video element displays nothing, and I don't get any errors back. I am using Chrome Version 51.0.2704.103 (64-bit) on Elementary OS (Ubuntu 14.04 based linux OS).

As a sidenote, if I record all the blobs into an array and then create a new blob and set the video's src element to URL.createObjectUrl(blob), it displays video correctly.

Here is the code I tried to accomplish this (minus the relaying, I'm just trying to play it locally):

var ms = new MediaSource();
var video = document.querySelector("video"); 
video.src = window.URL.createObjectURL(ms);

ms.addEventListener("sourceopen", function() {
    var sourceBuffer = ms.addSourceBuffer('video/webm; codecs="vorbis,vp8"');

    navigator.getUserMedia({video: {width: 320, height: 240, framerate: 30}, audio: true}, function(stream) {
        var recorder = new MediaRecorder(stream);

        recorder.ondataavailable = function(event) {
            var reader = new FileReader();
            reader.addEventListener("loadend", function () {
                var uint8Chunk = new Uint8Array(reader.result);
                if (!sourceBuffer.updating) {
                    sourceBuffer.appendBuffer(uint8Chunk);
                }
                if (video.paused) video.play();
            });
            reader.readAsArrayBuffer(event.data);
        };

        recorder.start(10);
    }, function(error) {
        console.error(error);
    });
}, false);

Here is the info I get in chrome://media-internal:

render_id: 147
player_id: 0
pipeline_state: kPlaying
event: WEBMEDIAPLAYER_CREATED
url: blob:http%3A//localhost%3A8080/e5c51dd8-5709-4e6f-9457-49ac8c34756b
found_audio_stream: true
audio_codec_name: opus
found_video_stream: true
video_codec_name: vp8
duration: unknown
audio_dds: false
audio_decoder: OpusAudioDecoder
video_dds: false
video_decoder: FFmpegVideoDecoder

Also the log:

00:00:00 00 pipeline_state  kCreated
00:00:00 00 event   WEBMEDIAPLAYER_CREATED
00:00:00 00 url blob:http%3A//localhost%3A8080/e5c51dd8-5709-4e6f-9457-49ac8c34756b
00:00:00 00 pipeline_state  kInitDemuxer
00:00:01 603    found_audio_stream  true
00:00:01 603    audio_codec_name    opus
00:00:01 603    found_video_stream  true
00:00:01 603    video_codec_name    vp8
00:00:01 604    duration    unknown
00:00:01 604    pipeline_state  kInitRenderer
00:00:01 604    audio_dds   false
00:00:01 604    audio_decoder   OpusAudioDecoder
00:00:01 604    video_dds   false
00:00:01 604    video_decoder   FFmpegVideoDecoder
00:00:01 604    pipeline_state  kPlaying

Update: I've tried sending the data to node and saving it to a webm file with ffmpeg (fluent-ffmpeg), and I can view the file in VLC correctly.

Update 2: After streaming it back from node, I get the following: Media segment did not contain any video coded frames, mismatching initialization segment. Therefore, MSE coded frame processing may not interoperably detect discontinuities in appended media.. After doing some research, it appears that webm files must be segmented to work, however I have not come across a way to do this (either using ffmpeg or other tools) for live streams. Any ideas here?

来源：https://stackoverflow.com/questions/37957007/display-getusermediastream-live-video-with-media-stream-extensions-mse

标签

webrtc

live-streaming

getusermedia

media-source