Capture RGB from Kinnect with Openni and show with OpenCV

Question

I need to capture the color RGB image from a Kinnect camera, but I want to show it in OpenCV as this is only a part of a bigger program. I know OpenCV has compatibility with Ope

Answer 1

Ok, first you should separate initialization from the loop for reading frames, this way.

Initialization

openni::Device device;  
openni::VideoStream  color;
openni::VideoFrameRef colorFrame;

rc = openni::OpenNI::initialize();
rc = device.open(openni::ANY_DEVICE);
rc = color.create(device, openni::SENSOR_COLOR);
rc = color.start();

Mat frame;

Now comes the main loop for reading frames. You have done almost everything, the only thing left is to copy the buffer to the openCV Mat.

Loop for reading frames

while (true)
{
   color.readFrame(&colorFrame);
   const openni::RGB888Pixel* imageBuffer = (const openni::RGB888Pixel*)colorFrame.getData();

   frame.create(colorFrame.getHeight(), colorFrame.getWidth(), CV_8UC3);
   memcpy( frame.data, imageBuffer, 3*colorFrame.getHeight()*colorFrame.getWidth()*sizeof(uint8_t) );

   cv::cvtColor(frame,frame,CV_BGR2RGB); //this will put colors right
}

Answer 2

Following up on @Jav_Rock's answer above, for IR and Depth the solution is similar, except for the following (in place of SENSOR_COLOR, CV_8UC3 and RGB888Pixel, respectively):

Depth

• Sensor type: SENSOR_DEPTH
• OpenCV type: CV_16UC1
• VideoFrameRef data: DepthPixel

Note you'll probably want to set the PixelMode for the VideoStream's VideoFormat to PIXEL_FORMAT_DEPTH_100_UM otherwise your depth image will look very black.

IR

• Sensor type: SENSOR_IR
• OpenCV type: CV_16UC1
• VideoFrameRef data: Grayscale16Pixel

Finally, note that neither depth nor IR need the cv::cvtColor call.