The coordinate system of pinhole camera model

Question

Recently, I have been studying the pinhole camera model, but I was confused with the model provided by OpenCV and the \"Multiple View geometry in computer vision\" tex

Answer 1

I finally figured out this issue and proved my interpretation is right by implementing the paper by Z. Zhang, Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision (ICCV'99), Corfu, Greece, pages 666-673, September 1999.

Let me explain everything from scratch. The following photo is the original pinhole camera model and the projected result on the image sensor. However, this is not what we are supposed to see on the "image".

What we should see is

Comparing figure 1 and 2, we should notice that this picture is up-side-down and left-side-right. My friend who works for a CMOS sensor company told me that there are built-in functions to automatically flip the perceived image.

Since we want to model the relationship between image coordinate and the world coordinate, we should directly treat the image sensor as a projection plane. What confused me previously is the projection is always limited to the projected side and this misled me to geometrically understand the derivation.

Now, we should look from the "back" of the image sensor as the blue (View Perspective) arrow.

The result is as figure 2. the x1-y1 coordinate is now toward right and down respectively, so the equations are

x1=-f(X/Z)
y1=-f(Y/Z)

Now, in terms of the x-y coordinate, the equation is

x=f(X/Z)+u0
y=f(Y/Z)+v0

which are what the paper described.

Now, let us take a look at the equivalent model which doesn't exist in real world but helps visual interpretation.

The principle is the same. Look from the center of projection and towards the image plane. The result is

where the projected "F" is right-side-left. The equations are

x1=f(X/Z)
y1=f(Y/Z)

Now, in terms of the x-y coordinate, the equation is

x=f(X/Z)+u0
y=f(Y/Z)+v0

which are what the paper described.

Last but not least, since the unit in world coordinate is mm or inch and the one in image coordinate is pixels, there is a scaling factor where some books describe as

x=a*f(X/Z)+u0 
y=b*f(Y/Z)+v0

or

x=fx(X/Z)+u0
y=fy(Y/Z)+v0

where fx=a*f, fy=b*f

Answer 2

It is actually much simpler: The coordinates of your object are supposed to be in camera world coordinates, which is a coordinate system whose x- and y- axis are parallel to the respective axis on the image plane, e.g. here: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT9/node2.html