3D Camera coordinates to world coordinates (change of basis?)

Question

Suppose I have the coordinates X, Y, Z and orientation Rx, Ry, Rz of an object with respect to a camera. In addition, I have the coordinates U, V, W and orientation Ru, Rv, Rw of this camera in the world.

How do I transform the position (location and rotation) of the object to its position in the world?

It sounds like a change of basis to me, but I haven't found a clear source yet.

Answer 1

In fact you have two basis : one relative to the camera, the other is absolute (the world). So you basically want to transform your relative data into absolute data.

Location

This is the easiest one. You have to translate the (X,Y,Z) position by the vector t(U,V,W). So all your positions in absolute are (Ax, Ay, Az) = (X,Y,Z)+t = (X+U,Y+V,Z+W).

Orientation

This is a bit more difficult. You have to find the rotation matrix that rotate your camera from (I assume) (0,0,1) to (Ru,Rv,Rw). You should look at Basic Rotation Matrices in order to decompose the 2 rotations that take (0,0,1) to (Ru,Rv,Rw) (one according to X axis, one according to Z axis for example). I advise you to draw the absolute basis and the vector (Ru,Rv,Rw) on a sheet of paper, it is the simplest way to get the right result.

So you have 2 basic rotations matrices r1 and r2. The resultant rotation matrix r = r1*r2 (or r2*r1, it doesn't matter). So the absolute orientation of your object is (ARx, ARy, ARz) = r*(Rx,Ry,Rz).

Hope this helps !

Answer 2

I have found this document which is quite clear on the topic. http://www.cse.psu.edu/~rcollins/CSE486/lecture12.pdf

It treats, among others, the reverse operation, i.e., going from world to camera 3D coordinates.

Pc = R ( Pw - C ) Where, Pc is a point in the camera world, Pw is a point in the normal world, R is a rotation matrix and C is the camera translation.

Unfortunately it is rather cumbersome to add latex formulae, so I will give some matlab code instead.

function lecture12_collins()
% for plotting simplicity I choose my points on plane z=0 in this example
% Point in the world
Pw = [2 2.5 0 1]';
% rotation
th = pi/3;
% translation
c = [1 2.5 0]';
% obtain world to camera coordinate matrix
T = GetT(th, c);
% calculate the camera coordinate
Pc = T*Pw
% get the camera to world coordinate
T_ = GetT_(th, c)
% Alternatively you could use the inverse matrix
% T_ = inv(R*C)

% calculate the worldcoordinate
Pw_ = T_*Pc

assert (all(eq(Pw_ ,Pw)))


function T = GetT(th, c)
% I have assumed rotation around the z axis only here.
R = [cos(th) -sin(th) 0 0
     sin(th)  cos(th) 0 0
          0        0  1 0
          0        0  0 1];
C = [1  0   0   -c(1)
     0  1   0   -c(2)
     0  0   1   -c(3)
     0  0   0   1];
 T = R*C;

function T_ = GetT_(th, c)
% negate the angle
R_ = [cos(-th) -sin(-th) 0 0
     sin(-th)  cos(-th) 0 0
          0        0  1 0
          0        0  0 1];
% negate the translation
C_ = [1  0   0   c(1)
     0  1   0   c(2)
     0  0   1   c(3)
     0  0   0   1];
T_ = C_*R_

So far this is about the location only. The rotation I've solved by using extra knowledge I had of the rotation. I know that my camera is perpendicular to the object and that its rotation is only around the z axis. I can just add the rotation of the camera and the object.