OpenCV create Mat from byte array

Question

In my C++ dll I am creating Mat from byte array:

BYTE * ptrImageData;  //Image data is in this array passed to this function

Mat newImg = Mat(nImageHeight,          


        

Answer 1

The C++ code appears ok, in that this creates a matrix wrapper for the supplied image data, assuming the buffer is in the conventional RGB8 format. Note that this constructor does not copy the buffer, so the buffer must remain valid for the duration of this Mat instance (or be copied).

Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData);

It appears the problem lies in Your C# code. I am not a C# developer, but I will do my best to help. You are creating a memory stream and using the JPEG codec to write a compressed version of the image into the buffer as if it were a file. But that is not the data format that cv::Mat is expecting, so you will basically see garbage (compressed data interpreted as uncompressed).

Given a System.Image.Drawing.Image instance, you can create a wrapper Bitmap object directly (or maybe use as, since it is a simple downcast). Then you can just use the Bitmap.LockBits() method tog obtain a pointer to the underlying image data.

Bitmap bmp = new Bitmap(sourceImage);

// Lock the bitmap's bits.  
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
System.Drawing.Imaging.BitmapData bmpData =
    bmp.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite,
    bmp.PixelFormat);

// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;

// Declare an array to hold the bytes of the bitmap.
int bytes  = Math.Abs(bmpData.Stride) * bmp.Height;
byte[] rgbBuffer = new byte[bytes];

// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, rgbBuffer, 0, bytes);

// Do your OpenCV processing...
// ...

// Unlock the bits.
bmp.UnlockBits(bmpData);

and then you can pass the rgbBuffer to OpenCV.

I'm not convinced that the memory management in the original code is entirely correct either, but anyway the above will work provided the scope of the buffer ownership is within the lock and unlock method calls. If the image data is to outlive this code block, you will have to copy the buffer.

Be careful with your pixel formats too - you need to make sure the Image/Bitmap instance really contains RGB8 data. OpenCV's cv::Mat has various flags so you can work with a variety of in-memory image formats. But note that these are not the same as the on-disk (typically compressed) formats, such as PNG, TIFF, and so forth.