OpenCV create Mat from byte array

Question

In my C++ dll I am creating Mat from byte array:

BYTE * ptrImageData;  //Image data is in this array passed to this function

Mat newImg = Mat(nImageHeight,          


        

Answer 1

Yes, this is one way to create a Mat from a byte array. You just have to be careful that your array contains what you think it does.

The image is created with some gray shade not the original one.

So you are getting an image in newImg? What was the pixel format of the original data?

Maybe you've switched the red and blue channels. The following line will swap the channels:

cv::cvtColor(newImg,swappedImg,CV_RGB2BGR);

Answer 2

The C++ code appears ok, in that this creates a matrix wrapper for the supplied image data, assuming the buffer is in the conventional RGB8 format. Note that this constructor does not copy the buffer, so the buffer must remain valid for the duration of this Mat instance (or be copied).

Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData);

It appears the problem lies in Your C# code. I am not a C# developer, but I will do my best to help. You are creating a memory stream and using the JPEG codec to write a compressed version of the image into the buffer as if it were a file. But that is not the data format that cv::Mat is expecting, so you will basically see garbage (compressed data interpreted as uncompressed).

Given a System.Image.Drawing.Image instance, you can create a wrapper Bitmap object directly (or maybe use as, since it is a simple downcast). Then you can just use the Bitmap.LockBits() method tog obtain a pointer to the underlying image data.

Bitmap bmp = new Bitmap(sourceImage);

// Lock the bitmap's bits.  
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
System.Drawing.Imaging.BitmapData bmpData =
    bmp.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite,
    bmp.PixelFormat);

// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;

// Declare an array to hold the bytes of the bitmap.
int bytes  = Math.Abs(bmpData.Stride) * bmp.Height;
byte[] rgbBuffer = new byte[bytes];

// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, rgbBuffer, 0, bytes);

// Do your OpenCV processing...
// ...

// Unlock the bits.
bmp.UnlockBits(bmpData);

and then you can pass the rgbBuffer to OpenCV.

I'm not convinced that the memory management in the original code is entirely correct either, but anyway the above will work provided the scope of the buffer ownership is within the lock and unlock method calls. If the image data is to outlive this code block, you will have to copy the buffer.

Be careful with your pixel formats too - you need to make sure the Image/Bitmap instance really contains RGB8 data. OpenCV's cv::Mat has various flags so you can work with a variety of in-memory image formats. But note that these are not the same as the on-disk (typically compressed) formats, such as PNG, TIFF, and so forth.

Answer 3

Here is link to docs: http://docs.opencv.org/modules/core/doc/basic_structures.html#mat-mat

In general you should take care about two things:

1. When you pass external data into matrix constructor, the external data is not automatically deallocated, so you should take care of it. If you want OpenCV matrix to care about memory, then you should copy matrix (you can do it in many ways, e.g. using Mat::clone or Mat::copyTo methods.
2. External data may not be continuous, i.e. size of row may be bigger than width multiplied by number of channels multiplied by size of data element. So you may want specify "step" as last argument of constructor. If you allocate external data manually and 100% sure that it is continuous, then you may not pass step and rely on automatic step calculation.

I am not familiar with C#, but it seems to me that you release data right after ProcessImage call. So if ProcessImage is asynchronous or somehow caches your matrix (i.e. lifetime of matrix is longer that ProcessImage call), then you should care about memory management.