OpenCV warping image based on calcOpticalFlowFarneback

Question

I\'m trying to perform a complex warp of an image using Dense Optical Flow (I am trying to wap the second image into roughly the same shape as the first image). I\'m probably g

Answer 1

The remap function can't work with optical flow. The function remap transforms the source image using the specified map:

dst(x, y) = src(mapx(x, y), mapy(x, y))

Optical flow has another formula:

frame1(x, y) = frame2(x + flowx(x, y), y + flowy(x, y))

So to use remap function first you need to create a map from flow:

Mat flow; // backward flow
calcOpticalFlowFarneback(nextFrame, prevFrame, flow);

Mat map(flow.size(), CV_32FC2);
for (int y = 0; y < map.rows; ++y)
{
    for (int x = 0; x < map.cols; ++x)
    {
        Point2f f = flow.at<Point2f>(y, x);
        map.at<Point2f>(y, x) = Point2f(x + f.x, y + f.y);
    }
}

Mat newFrame;
remap(prevFrame, newFrame, map);