How do you redistort a point?

问题

I have a distorted image on which I use Cv2.Undistort() which straightens it out. After this I find some points in the image. I need to get the location of those points in the distorted image.

I have tried Cv2.ProjectPoints() but failed to get appropriate coordinates. They were outside the bounds of the image.

This is how I went about doing this:

List<float> arr = new List<float>();
foreach (var i in points)
{
    arr.Add(i.X);
    arr.Add(i.Y);
    arr.Add(0);
}
Mat pointArr = new Mat(3, points.Count, MatType.CV_32FC1, arr.ToArray());
float[] rotArr = {  1, 0, 0,
                    0, 1, 0,
                    0, 0, 1};

float[] transArr = { 0, 0, 0 };

Mat rot = new Mat(3, 1, MatType.CV_32F, rotArr);
Mat trans = new Mat(3, 1, MatType.CV_32F, transArr);


Cv2.ProjectPoints(pointArr, rot, trans, camMatrix, dist, outputPoints);
List<Point2f> point2Fs = new List<Point2f>();
var lngt = outputPoints.Rows * outputPoints.Cols * outputPoints.Channels();
var matarr = new List<float>();
for (int i = 0; i < outputPoints.Rows; i++)
{
    point2Fs.Add(new Point2f(outputPoints.ExtractChannel(0).At<float>(0, i), outputPoints.ExtractChannel(1).At<float>(0, i)));
}

points - the points in the undistorted image I want to find in the original

Any suggestions?

Thanks!

回答1:

Unfortunately, OpenCV doesn't provide a function to do directly what you want, which is to apply the distortion coefficients to undistorted 2D points. You can:

Option 1

You are on the right track using projectPoints(). The trick to understand how this can work is that a point in the undistorted image "lives" in the image plane, which looks like this (picture from https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html):

Where the image plane is located at f (focal distance) from the origin and x and y coordinates are relative to c_x and c_y.

What you have to do is to build your 3D points (x,y,z) from the undistorted 2D points (x_u, y_u) like this:

x = (x_u - c_x) / f_x
y = (y_u - c_y) / f_y
z = 1

Instead of using (x_u, y_u, 0) as in your first foreach loop.

Then call projectPoints().

You are correctly using an identity rotation and 0 translation. f_x, f_y, c_x and c_y, are in cameraMatrix:

|f_x  0  c_x|
| 0  f_y c_y|
| 0   0   1 |

Option 2:

You can directly apply the equations for the distortion (https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html):

In the first step, you replace x' and y' by:

x' = (x_u - c_x) / f_x
y' = (y_u - c_y) / f_y

Then apply the rest of the equations. This is what projectPoints() does, without the rotation/translation part.

回答2:

I ended up using 2nd option proposed by Milo. I have rewritten the project point method without the rotation and translation part and starting the equation path from x' and y', where:

x' = (x_u - c_x) / f_x
y' = (y_u - c_y) / f_y

Then I applied the rest of the provided equations.

Here is the source code:

Point2f DistortPoint(Point2f point)
{
   var tempPt = new Point2f((point.X-intr.cx)/intr.fx, (point.Y-intr.cy)/intr.fy);
   var r2 = Math.Pow(tempPt.X, 2) + Math.Pow(tempPt.Y, 2);
   var xtmp = tempPt.X * ((1 + intr.k1 * r2 + intr.k2 * Math.Pow(r2, 2) + intr.k3 * Math.Pow(r2, 3)) /
                (1 + intr.k4 * r2 + intr.k5 * Math.Pow(r2, 2) + intr.k6 * Math.Pow(r2, 3))) + 2 * intr.p1 * tempPt.X * tempPt.Y + intr.p2 * (r2 + 2 * Math.Pow(tempPt.X, 2));
   var ytmp = tempPt.Y * ((1 + intr.k1 * r2 + intr.k2 * Math.Pow(r2, 2) + intr.k3 * Math.Pow(r2, 3)) /
                (1 + intr.k4 * r2 + intr.k5 * Math.Pow(r2, 2) + intr.k6 * Math.Pow(r2, 3))) + 2 * intr.p2 * tempPt.X * tempPt.Y + intr.p1 * (r2 + 2 * Math.Pow(tempPt.Y, 2));
   return new Point2f((float)(intr.fx * xtmp + intr.cx), (float)(intr.fy * ytmp + intr.cy));
}

回答3:

I did this by use indexing mapx and mapy returned from cv2.initUndistortRectifyMap().

 x_distored = mapx[y_rectified, x_rectified]
 y_distored = mapy[y_rectified, x_rectified]

The limitation of this method is that the indices must be within the ranges.

I aslo have problem to use Cv2.ProjectPoints() to project rectified locations to distorted, even after normalizing x_rectified and y_rectified. My problem is unable to convert R and T (I got them from cv2.stereoRectify(). They are not all zeros.) to rvec and tvec required by Cv2.ProjectPoints() as input. If anyone has successful experience please share it, thanks!

来源：https://stackoverflow.com/questions/63376649/how-do-you-redistort-a-point