2D-3D homography matrix estimation

Question

I am working with my Kinect on some 2D 3D image processing. Here is my problem: I have points in 3D (x,y,z) which lie on a plane. I also know the coordinates of the points on the RGB image (x,y). Now I want to estimate a 2D-3D homography matrix to estimate the (x1,y1,z1) coordinates to a random (x1,y1) point. I think that is possible, but I don't know where to start.

Thanks!

Answer 1

What you're looking for is a camera projection matrix, not a homography. A homography maps a plane seen from a camera to the same plane seen from another.

For estimating the camera matrix, look up solutions to solving for camera pose from known point correspondences. To get you started, a seminal paper on the topic and the OpenCV implementation solvePnP.