optical flow .flo files

问题

I have a few questions for doing optical flow projects. I use Python 2 (planning to use lasagne to use deep learning to learn optical flow), and don't know how to convert the c++ functions to that of python in visualization of the flows.

I downloaded (from http://vision.middlebury.edu/flow/data/comp/zip/other-gt-flow.zip) some image pairs where I have to estimate their optical flow, and their ground truth flow (.flo file). The problem is, when I read the .flo file into the program, it is a vectorized code. How do I view them like how they show in the webpage (http://vision.middlebury.edu/flow/data/)? I read from various sources and tried the following, but doesn't work.
In evaluating EPE (end point error) in what form should I have my prediction to be compared with the .flo file?

The code:

################################ Reading flow file ################################

f = open('flow10.flo', 'rb')

x = np.fromfile(f, np.int32, count=1) # not sure what this gives
w = np.fromfile(f, np.int32, count=1) # width
h = np.fromfile(f, np.int32, count=1) # height
print 'x %d, w %d, h %d flo file' % (x, w, h)

data = np.fromfile(f, np.float32) # vector 

data_2D = np.reshape(data, newshape=(388,584,2)); # convert to x,y - flow
x = data_2D[...,0]; y = data_2D[...,1]; 

################################ visualising flow file ################################
mag, ang = cv2.cartToPolar(x,y)
hsv = np.zeros_like(x)
hsv = np.array([ hsv,hsv,hsv ])
hsv = np.reshape(hsv, (388,584,3)); # having rgb channel
hsv[...,1] = 255; # full green channel
hsv[...,0] = ang*180/np.pi/2 # angle in pi
hsv[...,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX) # magnitude [0,255]
bgr = cv2.cvtColor(hsv,cv2.COLOR_HSV2BGR)
bgr = draw_hsv(data_2D)
cv2.imwrite('opticalhsv.png',bgr)

回答1:

On Middlebury's page there is a zip file called flow-code (http://vision.middlebury.edu/flow/code/flow-code.zip), which provides a tool called color_flow to convert those .flo files to color images.

On the other hand, if you want to implement your own code to do the transformation, i have this piece of code (i cannot provide the original author, it has been some time) that helps you to first compute the color:

static Vec3b computeColor(float fx, float fy)
{
static bool first = true;

// relative lengths of color transitions:
// these are chosen based on perceptual similarity
// (e.g. one can distinguish more shades between red and yellow
//  than between yellow and green)
const int RY = 15;
const int YG = 6;
const int GC = 4;
const int CB = 11;
const int BM = 13;
const int MR = 6;
const int NCOLS = RY + YG + GC + CB + BM + MR;
static Vec3i colorWheel[NCOLS];

if (first)
{
    int k = 0;

    for (int i = 0; i < RY; ++i, ++k)
        colorWheel[k] = Vec3i(255, 255 * i / RY, 0);

    for (int i = 0; i < YG; ++i, ++k)
        colorWheel[k] = Vec3i(255 - 255 * i / YG, 255, 0);

    for (int i = 0; i < GC; ++i, ++k)
        colorWheel[k] = Vec3i(0, 255, 255 * i / GC);

    for (int i = 0; i < CB; ++i, ++k)
        colorWheel[k] = Vec3i(0, 255 - 255 * i / CB, 255);

    for (int i = 0; i < BM; ++i, ++k)
        colorWheel[k] = Vec3i(255 * i / BM, 0, 255);

    for (int i = 0; i < MR; ++i, ++k)
        colorWheel[k] = Vec3i(255, 0, 255 - 255 * i / MR);

    first = false;
}

const float rad = sqrt(fx * fx + fy * fy);
const float a = atan2(-fy, -fx) / (float)CV_PI;

const float fk = (a + 1.0f) / 2.0f * (NCOLS - 1);
const int k0 = static_cast<int>(fk);
const int k1 = (k0 + 1) % NCOLS;
const float f = fk - k0;

Vec3b pix;

for (int b = 0; b < 3; b++)
{
    const float col0 = colorWheel[k0][b] / 255.f;
    const float col1 = colorWheel[k1][b] / 255.f;

    float col = (1 - f) * col0 + f * col1;

    if (rad <= 1)
        col = 1 - rad * (1 - col); // increase saturation with radius
    else
        col *= .75; // out of range

    pix[2 - b] = static_cast<uchar>(255.f * col);
}

return pix;
}

Then it calls the above function for all the pixels:

static void drawOpticalFlow(const Mat_<Point2f>& flow, Mat& dst, float maxmotion = -1)
{
dst.create(flow.size(), CV_8UC3);
dst.setTo(Scalar::all(0));

// determine motion range:
float maxrad = maxmotion;

if (maxmotion <= 0)
{
    maxrad = 1;
    for (int y = 0; y < flow.rows; ++y)
    {
        for (int x = 0; x < flow.cols; ++x)
        {
            Point2f u = flow(y, x);

            if (!isFlowCorrect(u))
                continue;

            maxrad = max(maxrad, sqrt(u.x * u.x + u.y * u.y));
        }
    }
}

for (int y = 0; y < flow.rows; ++y)
{
    for (int x = 0; x < flow.cols; ++x)
    {
        Point2f u = flow(y, x);

        if (isFlowCorrect(u))
            dst.at<Vec3b>(y, x) = computeColor(u.x / maxrad, u.y / maxrad);
    }
}
}

This is for my use in OpenCV, but the code help should anyone who wants achieve something similar.

来源：https://stackoverflow.com/questions/40330965/optical-flow-flo-files

标签

python

OpenCV

deep-learning

opticalflow