Google Tango: Aligning Depth and Color Frames

Question

I would like to align a (synchronous) depth/color frame pair, using the Google Tango tablet, such that, assuming that both frames have the same resolution, each pixel in the

Answer 1

I have not tried this but we can probably do: for each (X,Y,Z) from point cloud:

u_pixel = -(X/Z)* Fx, v_pixel = -(Y/Z)* Fy.
x = (u-cx)/Fx, y = (v-cy)/Fy.

for distortion correction (k1,k2,k2 can from distortion[] part of TangoInstrinsics, r = Math.sqrt(x^2 + y^2)))

x_corrected = x * (1 + k1 * r2 + k2 * r4 + k3 * r6) 
y_corrected = y * (1 + k1 * r2 + k2 * r4 + k3 * r6) 

Then we can convert normalized x_corrected, y_corrected to x_raster, y_raster by using reverse of the above formula (x_raster = x_correct*Fx+ cx)