Rendering Canvas context in Worker thread for smooth playback

问题

I have two videos here showing the rendering of a decoded MJPEG video sequence this one only rending one video:

Video description: Left (Source), Middle (Canvas copy of stream), Right (Decoded from network).

At this point the video is smooth both from source to network and back (websocket). And at least up to 5 video decoded and rendered is reasonably smooth. However if I render like 20 videos things start to lag:

My question is what is the best algorithm that will allow to render (or in multi-thread context) faster. Here's my code:

        socket.onmessage = function (m) {
            let blob = m.data;
            videoDecoder.postMessage(blob);
            videoDecoder2.postMessage(blob);
            videoDecoder3.postMessage(blob);
            // and so on... to 20
        }

For the purpose of testing, I just post all blobs to different video decoder web workers. Now for the rendering part:

  videoDecoder.onmessage = async function (e) {
      await renderImage(dCtx2, e.data);
  };

So in my code I have 20 of these onmessage handler (again for simplicity it's just copy and paste code.)

  async function renderImage(ctx, blob) {
    const isChrome = !!window.chrome && (!!window.chrome.webstore || !!window.chrome.runtime);
    if(isChrome) {
      const blobURL = URL.createObjectURL(blob);
      const img = new Image();
      img.onload = function() {
        ctx.drawImage(img, 0, 0);
        URL.revokeObjectURL(blobURL);
      };
      img.src = blobURL;
    } else {
      const bmp = await createImageBitmap(blob);
      ctx.drawImage(bmp, 0, 0);
      bmp.close();
    }
  }

Since the video decoders are basically Web Workers so running on each different thread so it's not the cause of drop of FPS in the canvas "players" in the UI. The renderImage method here is the main cause and it is the one that lags the main thread (the UI thread). What can be done to make each canvas context render on web workers perhaps so regardless how many players I have here it wont affect each other?

UPDATE:

I moved the rendering to web workers, as such passing the context down the web worker:

videoDecoders[i] = new Worker("videoDecoder.js");
const canvas = document.querySelector("#canvas" + (i+1)); // get the canvas
const offscreen = canvas.transferControlToOffscreen();
videoDecoders[i].postMessage({ action: 'init', canvas: offscreen }, [offscreen] );

Then basically it is just similar render image to the canvas but within the web worker thread.

And it did not help with making the playback smooth.

来源：https://stackoverflow.com/questions/61336406/rendering-canvas-context-in-worker-thread-for-smooth-playback