Modify canvas from wasm

Question

Is it possible to efficiently modify the html5 canvas from web assembly?

Update:

var imageData = context.getImageData(x, y, w, h)
var buffer = imageData         


        

Answer 1

WebAssembly instances typically have a linear memory area which is exposed to the JavaScript API as an arraybuffer. This can either be allocated in JS and passed in when the WebAssembly instance is created, or the WebAssembly instance can create it and export it to the JS code. Either way the arraybuffer can be used to efficiently copy data into and out of a Canvas element (using createImageData, getImageData and putImageData).

Answer 2

No, not in this stage of WebAssembly and web-api development. With context.getImageData you get a new ImageData object with a new buffer which must be copied once again in the memory's buffer of your WebAssembly instance. But if you don't need to read from canvas, only to write, you can allocate the ImageData.data in the memory of your WebAssembly instance. Use ImageData constructor

imageData = new ImageData(new Uint8ClampedArray(waInstance.export.memory.buffer, byteOffset, width*height*4), width, height)

imageData has a pointer to your data. On every rendering, do your work in WebAssembly and use the same imageData in context.putImageData(imageData), doing only once a big data copy per cycle.

Answer 3

I understand a solution has been accepted, but in case someone else lands here looking for an alternative I'll post anyways.

I don't actively participate in the development of the wasm-bindgen tool for rust, but its currently able to modify the canvas element through the web-sys crate. The code shown below is taken from the link on the wasm-bindgen book page.

use std::f64;
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsCast;

#[wasm_bindgen(start)]
pub fn start() {
    let document = web_sys::window().unwrap().document().unwrap();
    let canvas = document.get_element_by_id("canvas").unwrap();
    let canvas: web_sys::HtmlCanvasElement = canvas
        .dyn_into::<web_sys::HtmlCanvasElement>()
        .map_err(|_| ())
        .unwrap();

    let context = canvas
        .get_context("2d")
        .unwrap()
        .unwrap()
        .dyn_into::<web_sys::CanvasRenderingContext2d>()
        .unwrap();

    context.begin_path();

    // Draw the outer circle.
    context
        .arc(75.0, 75.0, 50.0, 0.0, f64::consts::PI * 2.0)
        .unwrap();

    // Draw the mouth.
    context.move_to(110.0, 75.0);
    context.arc(75.0, 75.0, 35.0, 0.0, f64::consts::PI).unwrap();

    // Draw the left eye.
    context.move_to(65.0, 65.0);
    context
        .arc(60.0, 65.0, 5.0, 0.0, f64::consts::PI * 2.0)
        .unwrap();

    // Draw the right eye.
    context.move_to(95.0, 65.0);
    context
        .arc(90.0, 65.0, 5.0, 0.0, f64::consts::PI * 2.0)
        .unwrap();

    context.stroke();
}

The canvas object is accessible from the rust code that is converted into web-assembly. There are many ways to invoke the web-assembly code, but the suggested one for this example is an index.js file with these contents, and a bundler like webpack.

import("path/to/wasm/canvas/code").catch(console.error)

For end to end demonstration of this follow this link as reference.

canvas hello world

Answer 4

1. Anyway, you will have to copy pixels to WebAssembly.Memory instance. Then modify and copy back.
2. I don't know why, but Uint8Array.set() is not fast in latest Chrome. It's better to recast data to 32 bit (new Uint32Array(your_uint8array)), and then use Uint32Array.set() to copy.
3. Keep in mind, that canva's .getImageData()/.setImageData() are not fast. Probably, because they do alpha premultiply and other things.

To summarize things: your most speed loss will be in .getImageData/.setImageData, and that can't be avoided. Other things have workarounds.

If compare with optimized JS, wasm will give you 10-20% of benefits, not too much.