3D rendering in OpenCV

Question

I am doing a project on 3D rendering of a scene. I am using OpenCV. The steps I am doing are like this:

1. Taking two images of a scene.
2. Calculating object

Answer 1

you can have Q matrix from stereo rectification via openCV method:

cv::stereoRectify

I think you want the Q matrix to reconstruct the 3D. However, you can reconstruct from intrinsic parameters via:

X = (u-cu)*base/d
Y = (v-cv)*base/d
Z = f*base/d 

where (u,v) is a 2D point in the image coordinate system, (cu,cv) is the principal point of the camera, f is the focal length, base is the baseline, d is the disparity and (X,Y,Z) is a 3D point in the camera coordinate system.

Answer 2

For the 3D part, you can render your scene with OpenGL or with PCL. You've two solutions:

• For each pixel, you make a point with the right color extracted from the camera's image. This will give you a point cloud which can be processed with PCL (for 3D features extraction for example).
• You apply a triangulation algorithm, but in order to apply this algorithm you must have the extrinsic matrices of your camera.

You can find more information about these techniques here:

• Point Cloud technique
• Triangulation algorithm

If you want to use OpenGL, you have to open a valid OpenGL context. I recommend you the SFML library or Qt. These libraries are very easy to use and have a good documentation. Both have tutorials about 3D rendering with OpenGL.

Answer 3

For the visualization, it is possible to use PCL or VTK (the visualization of PCL is based on vtk, but for me more simple to implement).

If you just want to have a look to the output you can just use some software like Meshlab

Cheers