tf.browser.fromPixel(video) causes memory leak

问题

I wanted to get the current frame data from a video using tf.browser.fromPixels(video), but every time this function is called, there is a memory leak. TensorFlow.js version

1.3.1 Browser version Chrome Version 75.0.3770.142 Here is the code that is causing the memory leak.

async function drawScreen () {

    console.log(tf.memory());

    var frame = tf.tidy( () => {

    return  tf.browser.fromPixels (videoElement, 3).expandDims(0).toFloat().div(tf.scalar(255))

    });

    console.log(tf.memory());

    hr_image_0 = tf.tidy( () => {

    return net.execute(frame).squeeze().clipByValue(0,1);

    });

    tf.dispose(frame);

    console.log(tf.memory());

    tf.browser.toPixels(hr_image_0, ccanvas);

    await tf.tidy( () =>  {

     tf.browser.toPixels(hr_image_0, ccanvas);

    });

    console.log(tf.memory());

}

I used

requstAnimationFrame()

to call the drawScreen function. Running the code on Chrome, the following is printed on the console

Object { unreliable: false, numBytesInGPU: 210113944, numTensors: 172,numDataBuffers: 172, numBytes: 211660580 }

CH6EX8.html:129:11 Object {unreliable: false, numBytesInGPU: 210344344, numTensors: 173,nnumDataBuffers: 173, numBytes: 211833380 }

CH6EX8.html:135:11 Object {unreliable: false, numBytesInGPU: 212187544, numTensors: 173, numDataBuffers: 173, numBytes: 213215780 }

CH6EX8.html:141:11 Object { unreliable: false, numBytesInGPU: 213745048, numTensors: 173, numDataBuffers: 173, numBytes: 213215780 }

Is there any way to remove the leak.

回答1:

According to the doc:

Executes the provided function fn and after it is executed, cleans up all intermediate tensors allocated by fn except those returned by fn

tf.tidy does remove intermediate tensor. But in this case there is not any intermediate tensor to remove. And the buffer will keep increasing as you keep calling the drawscreen method which will always return tf.browser.fromPixels (videoElement, 3).expandDims(0).toFloat().div(tf.scalar(255)) thus keeping the buffer allocated to the creation of the tensor in memory.

To clean the unused tensor, the method calling drawScreen should be enclosed in tf.tidy to dispose the tensor used when creating the tensor from image.

Since drawScreen is called by requestAnimation, tf.tidy can be a toplevel function inside it.

async function drawScreen () {
  tf.tidy(() => {
   // do all operations here
  })
}

The other option would be to use tf.dispose to dispose frame and hr_image_0 after the image has been drawn to the screen

await tf.browser.toPixels(hr_image_0, ccanvas);
tf.dispose(hr_image_0)

来源：https://stackoverflow.com/questions/59143276/tf-browser-frompixelvideo-causes-memory-leak

标签

tensorflow.js