How to align kinect's depth image with color image

Question

The image produced by the color and depth sensor on the Kinect are slightly out of alignment. How can I transform them to make them line up?

Answer 1

One way to do this is to assume that the color and depth images have similar variations in them, and to cross-correlate the two images (or smaller versions of them).

• Pre-whiten the images to get at the underlying variations.
• Cross-correlate the pre-whitened images or smaller versions of them.
• The peak position of the cross-correlation will tell you the offset in x and y