camera2 output to Bitmap

Question

I'm trying to use Google Mobile Vision API with the camera2 module and I'm having a lot of trouble.

I'm using Google's android-Camera2Video example code as a base. I've modified it to include the following callback:

Camera2VideoFragment.java

OnCameraImageAvailable mCameraImageCallback;

public interface OnCameraImageAvailable {
    void onCameraImageAvailable(Image image);
}


ImageReader.OnImageAvailableListener mImageAvailable = new ImageReader.OnImageAvailableListener() {
    @Override
    public void onImageAvailable(ImageReader reader) {

        Image image = reader.acquireLatestImage();
        if (image == null)
            return;

        mCameraImageCallback.onCameraImageAvailable(image);

        image.close();
    }
};

That way any fragment including Camera2VideoFragment.java can get access to its images.

Now, The Barcode API only accepts Bitmap images, but I'm unable to convert YUV_420_888 to Bitmap. Instead, I changed the imageReader's file format to JPEG and ran the following conversion code:

    Image.Plane[] planes = image.getPlanes();
    ByteBuffer buffer = planes[0].getBuffer();
    buffer.rewind();
    byte[] data = new byte[buffer.capacity()];
    buffer.get(data);
    Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);

This worked but the framerate drop of feeding JPEG data to the imageReader was significant. I'm wondering if anyone has worked around this issue before.

Answer 1

A late answer but hopefully still helpful.

As Ezequiel Adrian on his Example has explained the conversion of YUV_420_888 into one of the supported formats (In his case NV21), you can do the similar thing to get your Bitmap output:

private byte[] convertYUV420888ToNV21(Image imgYUV420) {
// Converting YUV_420_888 data to YUV_420_SP (NV21).
byte[] data;
ByteBuffer buffer0 = imgYUV420.getPlanes()[0].getBuffer();
ByteBuffer buffer2 = imgYUV420.getPlanes()[2].getBuffer();
int buffer0_size = buffer0.remaining();
int buffer2_size = buffer2.remaining();
data = new byte[buffer0_size + buffer2_size];
buffer0.get(data, 0, buffer0_size);
buffer2.get(data, buffer0_size, buffer2_size);
return data;}

Then you can convert the result into Bitmap:

Bitmap bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);