Calibration of images to obtain a top-view for points that lie on a same plane

Question

Calibration:

I have calibrated the camera using this vision toolbox in Matlab. I used checkerboard images to do so. After calibration I get the cameraPa

Answer 1

Since you have the size of squares on the grid, then given 2 points that you know are connected by an edge of size E (in real world units), you can calculate their 3D position.

Taking the camera intrinsic matrix K and the 3D position C and the camera orientation matrix R, you can calculate a ray to each of the points p by doing:

D = R^T * K^-1 * p

Each 3D point is defined as:

P = C + t*D

and you have the constraint that ||P1-P2|| = E then it's a matter of solving for t1,t2 and finding the 3D position of the two points.

In order to create a top view, you can take the 3D points and project them using a camera model for that top view to generate a new image.

If all your points are on a single plane, it's enough to calculate the position of 3 points, and you can extrapolate the rest.

If your points are located on a plane that you know one coordinate of, you can do it simply for each point. For example, if you know that your camera is located at height h=C.z, and you want to find the 3D location of points in the frame, given that they are on the floor (z=0), then all you have to do is calculate the direction D as above, and then:

t=abs( (h-0)/D.z )

The 0 represent the height of the plane. Substitute for any other value for other planes.

Now that you have the value of t, you can calculate the 3D position of each point: P=C+t*D.

Then, to create a top view, create a new camera position and rotation to match your required projection, and you can project each point onto this camera's image plane. If you want a full image, you can interpolate positions and fill in the blanks where no feature point was present.

For more details, you can always read: http://www.robots.ox.ac.uk/~vgg/hzbook/index.html