问题
I am writing an app that takes the camera feed, converts it to rgb, in order to do some processing.
It works fine on the old camera implementation which uses NV21 Yuv format. The issue I am having is with the new Yuv format, YUV_420_888. The image is no longer converted correctly to RGB in the new Camera2 Api which sends YUV_420_888 yuv format instead of NV21 (YUV_420_SP) format.
Can someone please tell me how should I convert YUV_420_888 to RGB?
Thanks
回答1:
In my approach I use OpenCV Mat and script from https://gist.github.com/camdenfullmer/dfd83dfb0973663a7974
First of all you convert your YUV_420_888 Image to Mat with the code in the link above.
*mImage is my Image object which i get in ImageReader.OnImageAvailableListener
Mat mYuvMat = imageToMat(mImage);
public static Mat imageToMat(Image image) {
ByteBuffer buffer;
int rowStride;
int pixelStride;
int width = image.getWidth();
int height = image.getHeight();
int offset = 0;
Image.Plane[] planes = image.getPlanes();
byte[] data = new byte[image.getWidth() * image.getHeight() * ImageFormat.getBitsPerPixel(ImageFormat.YUV_420_888) / 8];
byte[] rowData = new byte[planes[0].getRowStride()];
for (int i = 0; i < planes.length; i++) {
buffer = planes[i].getBuffer();
rowStride = planes[i].getRowStride();
pixelStride = planes[i].getPixelStride();
int w = (i == 0) ? width : width / 2;
int h = (i == 0) ? height : height / 2;
for (int row = 0; row < h; row++) {
int bytesPerPixel = ImageFormat.getBitsPerPixel(ImageFormat.YUV_420_888) / 8;
if (pixelStride == bytesPerPixel) {
int length = w * bytesPerPixel;
buffer.get(data, offset, length);
if (h - row != 1) {
buffer.position(buffer.position() + rowStride - length);
}
offset += length;
} else {
if (h - row == 1) {
buffer.get(rowData, 0, width - pixelStride + 1);
} else {
buffer.get(rowData, 0, rowStride);
}
for (int col = 0; col < w; col++) {
data[offset++] = rowData[col * pixelStride];
}
}
}
}
Mat mat = new Mat(height + height / 2, width, CvType.CV_8UC1);
mat.put(0, 0, data);
return mat;
}
We have 1 channel YUV Mat. Define new Mat for BGR(not RGB yet) image:
Mat bgrMat = new Mat(mImage.getHeight(), mImage.getWidth(),CvType.CV_8UC4);
I just started learning OpenCV so propably this doesn't have to be 4-channel Mat and instead could be 3-channel but it works for me. Now I use convert color method to change my yuv Mat into bgr Mat.
Imgproc.cvtColor(mYuvMat, bgrMat, Imgproc.COLOR_YUV2BGR_I420);
Now we can do all the image processing like finding contours, colors, circles, etc. To print image back on screen we need to convert it to bitmap:
Mat rgbaMatOut = new Mat();
Imgproc.cvtColor(bgrMat, rgbaMatOut, Imgproc.COLOR_BGR2RGBA, 0);
final Bitmap bitmap = Bitmap.createBitmap(bgrMat.cols(), bgrMat.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(rgbaMatOut, bitmap);
I have all my image processing in seperate thread so to set my ImageView I need to do this on the UI thread.
runOnUiThread(new Runnable() {
@Override
public void run() {
if(bitmap != null) {
mImageView.setImageBitmap(bitmap);
}
}
});
回答2:
Camera2 YUV_420_888 to RGB Mat(opencv) in Java
@Override
public void onImageAvailable(ImageReader reader){
Image image = null;
try {
image = reader.acquireLatestImage();
if (image != null) {
byte[] nv21;
ByteBuffer yBuffer = mImage.getPlanes()[0].getBuffer();
ByteBuffer uBuffer = mImage.getPlanes()[1].getBuffer();
ByteBuffer vBuffer = mImage.getPlanes()[2].getBuffer();
int ySize = yBuffer.remaining();
int uSize = uBuffer.remaining();
int vSize = vBuffer.remaining();
nv21 = new byte[ySize + uSize + vSize];
//U and V are swapped
yBuffer.get(nv21, 0, ySize);
vBuffer.get(nv21, ySize, vSize);
uBuffer.get(nv21, ySize + vSize, uSize);
Mat mRGB = getYUV2Mat(nv21);
}
} catch (Exception e) {
Log.w(TAG, e.getMessage());
}finally{
image.close();// don't forget to close
}
}
public Mat getYUV2Mat(byte[] data) {
Mat mYuv = new Mat(image.getHeight() + image.getHeight() / 2, image.getWidth(), CV_8UC1);
mYuv.put(0, 0, data);
Mat mRGB = new Mat();
cvtColor(mYuv, mRGB, Imgproc.COLOR_YUV2RGB_NV21, 3);
return mRGB;
}
回答3:
Have you tried using this script? It's an answer posted by yydcdut on this question
https://github.com/pinguo-yuyidong/Camera2/blob/master/camera2/src/main/rs/yuv2rgb.rs
回答4:
Approximately 10 times faster than the mentioned "imageToMat"-Function above is this code:
Image image = reader.acquireLatestImage();
...
Mat yuv = new Mat(image.getHeight() + image.getHeight() / 2, image.getWidth(), CvType.CV_8UC1);
ByteBuffer buffer = image.getPlanes()[0].getBuffer();
final byte[] data = new byte[buffer.limit()];
buffer.get(data);
yuv.put(0, 0, data);
...
image.close();
回答5:
So I ran into the exact same problem, where I had a code that took the old YUV_420_SP format byte[] data from OnPreviewFrame() and converted it to RGB.
They key here is that the 'old' data in the byte[] is as YYYYYY...CrCbCrCbCrCb, and the 'new' data from the Camera2 API is divided into 3 planes: 0=Y, 1=Cb, 2=Cr., from where you can obtain each's byte[]s. So, all you need to do is reorder the new data as a single array that matches the 'old' format, which you can pass to your existing toRGB() functions:
Image.Plane[] planes = image.getPlanes(); // in YUV220_888 format
int acc = 0, i;
ByteBuffer[] buff = new ByteBuffer[planes.length];
for (i = 0; i < planes.length; i++) {
buff[i] = planes[i].getBuffer();
acc += buff[i].capacity();
}
byte[] data = new byte[acc],
tmpCb = new byte[buff[1].capacity()] , tmpCr = new byte[buff[2].capacity()];
buff[0].get(data, 0, buff[0].capacity()); // Y
acc = buff[0].capacity();
buff[2].get(tmpCr, 0, buff[2].capacity()); // Cr
buff[1].get(tmpCb, 0, buff[1].capacity()); // Cb
for (i=0; i<tmpCb.length; i++) {
data[acc] = tmpCr[i];
data[acc + 1] = tmpCb[i];
acc++;
}
..and now data[] is formatted just as the old YUV_420_SP.
(hope that it helps someone, despite the years passed..)
回答6:
Use Shyam Kumar's answer is not right for my phone, but Daniel Więcek's is right.I debug it, find planes[i].getRowStride() is 1216, planes[i].getPixelStride() is 2. While image width and height is both 1200.
Because my reputation is 3, so I cann't comment but post an answer.
来源：https://stackoverflow.com/questions/30510928/convert-android-camera2-api-yuv-420-888-to-rgb