HTML5 Video buffered attribute features

Question

I am designing a custom HTML5 video player. Thus, it will have its own custom slider to mimic the video progress, so I need to understand the entire buffering s

Answer 1

This is just a variation of this excellent answer https://stackoverflow.com/a/18624833/985454

I only made it work without any work required and added few perks. Everything is automatic.

• currently intended for full-screen video playback such as netflix or hbogo
• automatically creates the canvas
• auto-updates width to 100% viewport width
• works as a bookmarklet
• does not obstruct the view much (transparent, 2px tall)

function prepare() {
    console.log('prepare');

    _v = $('video')[0];
    _v.insertAdjacentHTML('afterend',
    `<canvas
        id="WowSuchName"
        height="1"
        style="
            position: absolute;
            bottom: 0;
            left: 0;
            opacity: 0.4;
        "></canvas>`);

    _c = WowSuchName
    _cx = _c.getContext('2d');

    window.addEventListener('resize', resizeCanvas, false);

    function resizeCanvas() {
        console.log('resize');
        _c.width = window.innerWidth;
    }
    resizeCanvas();
}

/// buffer viewer loop (updates about every 2nd frame)
function loop() {
    if (!window.WowSuchName) { prepare(); }

    var b = _v.buffered,  /// get buffer object
        i = b.length,     /// counter for loop
        w = _c.width,     /// cache canvas width and height
        h = _c.height,
        vl = _v.duration, /// total video duration in seconds
        x1, x2;           /// buffer segment mark positions

    /// clear canvas
    _cx.clearRect(0, 0, w, h);

    /// color for loaded buffer(s)
    _cx.fillStyle = '#888';

    /// iterate through buffers
    while (i--) {
        x1 = b.start(i) / vl * w;
        x2 = b.end(i) / vl * w;
        _cx.fillRect(x1, 0, x2 - x1, h);
    }

    /// draw cursor for position
    _cx.fillStyle = '#fff';
    x1 = _v.currentTime / vl * w;
    _cx.fillRect(x1-1, 0, 2, h);

    setTimeout(loop, 29);
}

loop();

And the code for bookmarklet is here

javascript:eval(atob("CmZ1bmN0aW9uIHByZXBhcmUoKSB7CiAgICBjb25zb2xlLmxvZygncHJlcGFyZScpOwoKICAgIF92ID0gJCgndmlkZW8nKVswXTsKICAgIF92Lmluc2VydEFkamFjZW50SFRNTCgnYWZ0ZXJlbmQnLAogICAgYDxjYW52YXMKICAgICAgICBpZD0iV293U3VjaE5hbWUiCiAgICAgICAgaGVpZ2h0PSIxIgogICAgICAgIHN0eWxlPSIKICAgICAgICAgICAgcG9zaXRpb246IGFic29sdXRlOwogICAgICAgICAgICBib3R0b206IDA7CiAgICAgICAgICAgIGxlZnQ6IDA7CiAgICAgICAgICAgIG9wYWNpdHk6IDAuNDsKICAgICAgICAiPjwvY2FudmFzPmApOwogICAgCiAgICBfYyA9IFdvd1N1Y2hOYW1lCiAgICBfY3ggPSBfYy5nZXRDb250ZXh0KCcyZCcpOwoKICAgIHdpbmRvdy5hZGRFdmVudExpc3RlbmVyKCdyZXNpemUnLCByZXNpemVDYW52YXMsIGZhbHNlKTsKCiAgICBmdW5jdGlvbiByZXNpemVDYW52YXMoKSB7CiAgICAgICAgY29uc29sZS5sb2coJ3Jlc2l6ZScpOwogICAgICAgIF9jLndpZHRoID0gd2luZG93LmlubmVyV2lkdGg7CiAgICB9CiAgICByZXNpemVDYW52YXMoKTsKfQoKLy8vIGJ1ZmZlciB2aWV3ZXIgbG9vcCAodXBkYXRlcyBhYm91dCBldmVyeSAybmQgZnJhbWUpCmZ1bmN0aW9uIGxvb3AoKSB7CiAgICBpZiAoIXdpbmRvdy5Xb3dTdWNoTmFtZSkgeyBwcmVwYXJlKCk7IH0KCiAgICB2YXIgYiA9IF92LmJ1ZmZlcmVkLCAgLy8vIGdldCBidWZmZXIgb2JqZWN0CiAgICAgICAgaSA9IGIubGVuZ3RoLCAgICAgLy8vIGNvdW50ZXIgZm9yIGxvb3AKICAgICAgICB3ID0gX2Mud2lkdGgsICAgICAvLy8gY2FjaGUgY2FudmFzIHdpZHRoIGFuZCBoZWlnaHQKICAgICAgICBoID0gX2MuaGVpZ2h0LAogICAgICAgIHZsID0gX3YuZHVyYXRpb24sIC8vLyB0b3RhbCB2aWRlbyBkdXJhdGlvbiBpbiBzZWNvbmRzCiAgICAgICAgeDEsIHgyOyAgICAgICAgICAgLy8vIGJ1ZmZlciBzZWdtZW50IG1hcmsgcG9zaXRpb25zCgogICAgLy8vIGNsZWFyIGNhbnZhcwovLyAgICAgX2N4LmZpbGxTdHlsZSA9ICcjMDAwJzsKLy8gICAgIF9jeC5maWxsUmVjdCgwLCAwLCB3LCBoKTsKICAgIF9jeC5jbGVhclJlY3QoMCwgMCwgdywgaCk7CgogICAgLy8vIGNvbG9yIGZvciBsb2FkZWQgYnVmZmVyKHMpCiAgICBfY3guZmlsbFN0eWxlID0gJyM4ODgnOwoKICAgIC8vLyBpdGVyYXRlIHRocm91Z2ggYnVmZmVycwogICAgd2hpbGUgKGktLSkgewogICAgICAgIHgxID0gYi5zdGFydChpKSAvIHZsICogdzsKICAgICAgICB4MiA9IGIuZW5kKGkpIC8gdmwgKiB3OwogICAgICAgIF9jeC5maWxsUmVjdCh4MSwgMCwgeDIgLSB4MSwgaCk7CiAgICB9CgogICAgLy8vIGRyYXcgY3Vyc29yIGZvciBwb3NpdGlvbgogICAgX2N4LmZpbGxTdHlsZSA9ICcjZmZmJzsKICAgIHgxID0gX3YuY3VycmVudFRpbWUgLyB2bCAqIHc7CiAgICBfY3guZmlsbFJlY3QoeDEtMSwgMCwgMiwgaCk7CgogICAgc2V0VGltZW91dChsb29wLCAyOSk7Cn0KCmxvb3AoKTsK"))

Answer 2

According to

• https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video
• https://developer.mozilla.org/en-US/docs/Web/API/TimeRanges
• https://developer.mozilla.org/en-US/docs/Web/API/TimeRanges.start

the buffered attribute holds information about all currently buffered time ranges. To my understanding, if a buffered portion is lost, it is removed from the object (in case that ever happens).

Esepcially the last link seems to be very useful for understanding the matter (since it offers a code sample) but keep in mind these are mozilla documents and support might be different in other browsers.

To answer your questions

Say the video starts. It continues upto 1:45 on its own (occasionally stalling perhaps, waiting for further data), after which I suddenly jump to 32:45. Now after some time, if I jump back to 1:27 (within the time range initially loaded and played through, before I made the jump), will it start playing immediately as it was already loaded before?

It should play immediately when jumping back unless the buffer of that portion was unloaded. I think it's very reasonable to assume that buffers or buffer ranges are unloaded at some point if the overall buffersize exceeds a certain volume.

Say I make 5 or 6 such jumps, each time waiting for a few seconds for some data to load after the jump. Does that mean the buffered object will have all those time ranges stored?

Yes, all buffered ranges should be readable through the attribute.

Will checking whether the buffered object has one time range starting at 0 (forget live streaming) and ending at the video duration length ensure tht the entire video resource has been loaded fully?

Yes, this is the code example in the last link. Apparently this is an applicable method of determining if the entire video has been loaded.

if (buf.start(0) == 0 && buf.end(0) == v.duration)

Answer 3

How video is buffered is browser implementation-dependent and therefore may vary from browser to browser.

Various browsers can use different factors to determine to keep or to discard a part of the buffer. Old segments, disk space, memory, and performance are typical factors.

The only way to know is to "see" what the browser has or is loading.

For example - in Chrome I played a few seconds then I skipped to about 30 seconds and you can see that it starts to load another part starting from that position.

(The buffer also seem to be bounded to key-frames so it is possible to decode the n-frames in that buffer. This means the buffer can start to load data a little before the actual position).

I supplied a demo video about 1 minute long - however, this is not long enough to do proper testing. Free free to supply video links that contain longer video (or please share if you want me to update the demo with this).

The main function will iterate through the buffered object on the video element. It will render all parts that exist to the canvas right below the video showing in red.

You can click (bit not drag) on this viewer to move the video to different positions.

/// buffer viewer loop (updates about every 2nd frame)
function loop() {

    var b = vid.buffered,  /// get buffer object
        i = b.length,      /// counter for loop
        w = canvas.width,  /// cache canvas width and height
        h = canvas.height,
        vl = vid.duration, /// total video duration in seconds
        x1, x2;            /// buffer segment mark positions

    /// clear canvas with black
    ctx.fillStyle = '#000';
    ctx.fillRect(0, 0, w, h);

    /// red color for loaded buffer(s)
    ctx.fillStyle = '#d00';

    /// iterate through buffers
    while (i--) {
        x1 = b.start(i) / vl * w;
        x2 = b.end(i) / vl * w;
        ctx.fillRect(x1, 0, x2 - x1, h);
    }

    /// draw info
    ctx.fillStyle = '#fff';

    ctx.textBaseline = 'top';
    ctx.textAlign = 'left';
    ctx.fillText(vid.currentTime.toFixed(1), 4, 4);

    ctx.textAlign = 'right';
    ctx.fillText(vl.toFixed(1), w - 4, 4);

    /// draw cursor for position
    x1 = vid.currentTime / vl * w;

    ctx.beginPath();
    ctx.arc(x1, h * 0.5, 7, 0, 2 * Math.PI);
    ctx.fill();

    setTimeout(loop, 29);
}

Answer 4

Although the accepted answer's description is excellent, I decided to update its code sample, for several reasons:

• The progress render task should really be fired only on a progress event.
• The progress render task is mixed up with some other tasks like drawing the timestamp and the playhead position.
• The code refers to several DOM elements by their IDs without using document.getElementById().
• The variable names were all obscured.
• I thought a forward for() loop was more elegant than a backward while() loop.

Note that I have removed the playhead and timestamp to keep the code clean, as this answer focusses purely on visualisation of the video buffer.

LINK TO ONLINE VIDEO BUFFER VISUALISER

Rewrite of accepted answer's loop() function:

function drawProgress(canvas, buffered, duration){
    // I've turned off anti-aliasing since we're just drawing rectangles.
    var context = canvas.getContext('2d', { antialias: false });
    context.fillStyle = 'blue';

    var width = canvas.width;
    var height = canvas.height;
    if(!width || !height) throw "Canvas's width or height weren't set!";
    context.clearRect(0, 0, width, height); // clear canvas

    for(var i = 0; i < buffered.length; i++){
        var leadingEdge = buffered.start(i) / duration * width;
        var trailingEdge = buffered.end(i) / duration * width;
        context.fillRect(leadingEdge, 0, trailingEdge - leadingEdge, height);
    }
}

Answer 5

1. Almost every browser saves the buffered data in cache for that session. The cache expires after the user goes away from that page. I don't think that the user will have to load the page each time he loads the video from a point where the video has been loaded. The user will face this issue only when the server is clearing out all the cache data. HTML5 video tag will support this, and will save the video upto the point till where it has been loaded.

2. It doesnot mean that the session has been lost, it means that either the object (if you are using flash player) is looking for some data from that particular point or the html5 video tag is having some issues either because of the INTERNET connection failure, or some other server errors.

3. The metadata is automatically loaded, untill you use this <audio preload="none"... this will make the browser not to download anything from server, you can use it as:
   <audio preload="auto|metadata|none"... If you use none, nothing is downloaded unless the user clicks play button, and metadata will download name, timing and other meta data from server, but not the file somehow, auto will start downloading as soon as the page loads.

I will always refer you to read some documentations by jQuery. As the jQuery will let you change and update the content using ajax API and will be helpfull too. Hope you succeed! Cheers.