Media Source Extensions appendBuffer of WebM stream in random order

Question

I am trying to achieve video downloading in parallel from multiple sources. However MSE appendBuffer method always fails when not following sequence order of video file.

I would like to append parts in random order and play video "as soon as possible". I was exploring SourceBuffer mode property as well as timestampOffset. None of those were helpful.

I am wondering if source webm file i have could be in "not supported format" for such a task (sequential approach works fine).

source video file

Thank you for any advices.

UPDATE: I tried to analyse well known example video file and i figured out that it is possible to append parts of it out of order. Seems like it is necessary to follow Cluster byte ranges:

<Cluster type="list" offset="4357">
  <Timecode type="uint" value="0"/>
  <SimpleBlock type="binary" size="7723" trackNum="1" timecode="0" presentationTimecode="0" flags="80"/>
  <SimpleBlock type="binary" size="5" trackNum="2" timecode="0" presentationTimecode="0" flags="80"/>
  ...
</Cluster>
<Cluster type="list" offset="16187">
  <Timecode type="uint" value="385"/>
  <SimpleBlock type="binary" size="5" trackNum="2" timecode="0" presentationTimecode="385" flags="80"/>
  <SimpleBlock type="binary" size="4968" trackNum="1" timecode="13" presentationTimecode="398" flags="80"/>
  ...
</Cluster>

Answer 1

After digging into webm format specification, compiling libwebm tools and studying DASH i finally figured out how to make MSE appendBuffer working in any order!

1. ffmpeg -i result.webm -g 10 -c:v libvpx resultClusters.webm (you can also use libvpx-vp9)
2. sample_muxer -i resultClusters.webm -o resultRepaired.webm
3. mse_json_manifest resultRepaired.webm

You will get on stdout something like:

{
  "type": "video/webm; codecs=\"vp8\"",
  "duration": 27771.000000,
  "init": { "offset": 0, "size": 258},
  "media": [
    { "offset": 258, "size": 54761, "timecode": 0.000000 },
    { "offset": 55019, "size": 166431, "timecode": 2.048000 },
    { "offset": 221450, "size": 49258, "timecode": 4.130000 },
    { "offset": 270708, "size": 29677, "timecode": 6.148000 },
    { "offset": 300385, "size": 219929, "timecode": 8.232000 },
    { "offset": 520314, "size": 25132, "timecode": 10.335000 },
    { "offset": 545446, "size": 180777, "timecode": 12.440000 },
    { "offset": 726223, "size": 76107, "timecode": 14.471000 },
    { "offset": 802330, "size": 376557, "timecode": 14.794000 },
    { "offset": 1178887, "size": 247138, "timecode": 16.877000 },
    { "offset": 1426025, "size": 78468, "timecode": 18.915000 },
    { "offset": 1504493, "size": 25614, "timecode": 20.991000 },
    { "offset": 1530107, "size": 368277, "timecode": 23.093000 },
    { "offset": 1898384, "size": 382847, "timecode": 25.097000 },
    { "offset": 2281231, "size": 10808, "timecode": 27.135000 }
  ]
}

Now all you have to do is firstly load metadata xhr.setRequestHeader("Range", "bytes=0-257"); and then in ANY ORDER all other segments. E.g. second segment range is 55019-221449 bytes.

Explanation:

The most important thing is ffmpeg reencoding with group of frames set to the size of cluster you would like to have. In this example i choose pretty low threshold (each 10 frames) but you can choose higher causing fewer clusters are generated (less items in "media" array).

After that you have to fix cues in the classic way (using sample_muxer from libwebm) and you are ready to go.

Tested on: Chrome 51, Firefox 47.