Kinect: From Color Space to world coordinates

Question

I am tracking a ball using the rgb data from kinect. After this I look up the corresponding depth data. Both of this is working splendid. Now I want to have the actual x,y,z wor

Answer 1

Check the CameraIntrinsics.

typedef struct _CameraIntrinsics
{
    float FocalLengthX;
    float FocalLengthY;
    float PrincipalPointX;
    float PrincipalPointY;
    float RadialDistortionSecondOrder;
    float RadialDistortionFourthOrder;
    float RadialDistortionSixthOrder;
}   CameraIntrinsics;

You can get it from ICoordinateMapper::GetDepthCameraIntrinsics.

Then, for every pixel (u,v,d) in depth space, you can get the coordinate in world space by doing this:

x = (u - principalPointX) / focalLengthX * d;
y = (v - principalPointY) / focalLengthY * d;
z = d;

For color space pixel, you need to first find its associated depth space pixel, which you should use ICoordinateMapper::MapCameraPointTodepthSpace. Since not all color pixel has its associated depth pixel (1920x1080 vs 512x424), you can't have the full-HD color point cloud.