H264 video works using src attribute. Same video fails using the MediaSource API (Chromium)

Question

http://www.youtube.com/html5 indicates that Google Chrome is compliant with MediaSource Extensions & H.264.

I make a simple test checking that my video is supported by Chromium, using the <video id='player' autoplay='true'> <source src='/test.mp4' type='video/mp4' /> </video>

The video plays smoothly.

A second alternative that also works fine consists in loading through AJAX the byte chain and converting the buffer to a URI object. Then asigning such URI to the (video) source.src attribute.

Finally I try to load the same video through AJAX and inject it into a MediaSource Buffer. It fails with the error 4. (Source Not supported).

The code used is similar to:

var mediaSource = new (window.MediaSource || window.WebKitMediaSource)();
window.video = document.getElementById('video1');
window.video.addEventListener("error", function onError(err) {
    alert("window.video error detected:");
    console.dir(window.video.error); window.worker.terminate();
}); 
window.video.pause();
window.video.src = URL.createObjectURL(mediaSource);
var onMediaSourceOpen = function (e) {
    mediaSource.removeEventListener('sourceopen', onMediaSourceOpen);
    window.videoSource = mediaSource.addSourceBuffer('video/mp4;codecs="avc1.4d001e,mp4a.40.2"');
    injectVideoIntoBuffer();
}   

mediaSource.addEventListener('sourceopen', onMediaSourceOpen);

var injectVideoIntoBuffer = function onResponse() {
    var xhr = new XMLHttpRequest();
    xhr.open('GET', "test.mp4");
    xhr.responseType = 'arraybuffer';
    xhr.addEventListener("readystatechange", function () {
         // Next line raises a MediaError, code: 4, (MEDIA_ERR_SRC_NOT_SUPPORTED)
         videoSource.appendBuffer(new Uint8Array(xhr.response));
          ... 
    }, false);
    xhr.send();
}

I tried different mp4 files, generated either with ffmpeg/avconv or MP4Box. Not luck at this moment. A similar code works fine with VP8/WebM test files.

Thanks in advance for any help/hint or link!

Enrique

Answer 1

Thanks all for your answers. Looks like newer versions of Chrome fix the problem.

I wrongly assumed that if a codec is supported by the browser, it will automatically be supported by MSE. In practice, that's not the case. A browser can support a set of video codecs (h264/webM/theora/...), it can also support MSE, but just a subset of video codecs when "injecting" the video into MSE buffers.

The compatibility matrix between MSE and codecs doesn't only depends on the browser but also on the OS. So for example, Google Chrome Supports MSE+h264 on Windows and Android but not (yet?) on Linux. VP9+MSE is supported on Windows and Linux but not on Android.

Youtube has a very useful test page to check browser support for MSE & h264/VP9 codecs:

https://www.youtube.com/html5

Answer 2

Try this:

var injectVideoIntoBuffer = function onResponse() {
   var xhr = new XMLHttpRequest();
   xhr.open('GET', "test.mp4");
   xhr.responseType = 'arraybuffer';
   xhr.addEventListener("readystatechange", function () {
       if (xhr.readyState == xhr.DONE) {
           videoSource.appendBuffer(new Uint8Array(xhr.response));
       }
      ... 
   }, false);

It could be that you're only appending a fragment of the mp4. This would be fine if the fragment from the AJAX request is in whole mp4 atoms, i.e moov, moof, mdat. But i'm thinking that might not be the case.

If it still fails, you could try to transcode the movie again with: (NOTE! This will remove the sound)

ffmpeg -an -codec:v libx264 -profile:v baseline -level 3 -b:v 2000k -i in.mp4 out.mp4

and MP4Box with:

MP4Box -dash 10000 -rap -frag-rap out.mp4

Just to see if the movie works.

Answer 3

I think Chrome might only support WebM with media source extensions for now.