How to use ScriptIntrinsicYuvToRGB (converting byte[] yuv to byte[] rgba)

Question

I have byte[] yuvByteArray (540x360 image captured from Camera.PreviewCallback.onPreviewFrame method and dumped into assets/yuv.bin fi

Answer 1

Kotlin

        val rs = RenderScript.create(CONTEXT_HERE)
        val yuvToRgbIntrinsic = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs))

        val yuvType = Type.Builder(rs, Element.U8(rs)).setX(byteArray.size)
        val inData = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT)

        val rgbaType = Type.Builder(rs, Element.RGBA_8888(rs)).setX(width).setY(height)
        val outData = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT)

        inData.copyFrom(byteArray)

        yuvToRgbIntrinsic.setInput(inData)
        yuvToRgbIntrinsic.forEach(outData)

        val bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
        outData.copyTo(bitmap)

Answer 2

Our internal test app uses the following sequence to create the YUV allocation.

    tb = new Type.Builder(mRS, Element.createPixel(mRS, 
              Element.DataType.UNSIGNED_8, Element.DataKind.PIXEL_YUV));
    tb.setX(mWidth);
    tb.setY(mHeight);
    tb.setYuvFormat(android.graphics.ImageFormat.NV21);
    mAllocationIn = Allocation.createTyped(mRS, tb.create(), Allocation.USAGE_SCRIPT);

Then inside the callback that new YUV data is available do

    mAllocationIn.copyFrom(theYuvByteArray);

Answer 3

yuv.bin file is definitely in NV21 format, as it captures here http://developer.android.com/reference/android/hardware/Camera.PreviewCallback.html#onPreviewFrame

setYuvFormat method is from API level 18, I removed it

So this code works fine:

rs = RenderScript.create(this);
yuvToRgbIntrinsic = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));

Type.Builder yuvType = new Type.Builder(rs, Element.U8(rs)).setX(yuvByteArray.length);
Allocation in = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT);

Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs)).setX(W).setY(H);
Allocation out = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT);

in.copyFrom(yuvByteArray);

yuvToRgbIntrinsic.setInput(in);
yuvToRgbIntrinsic.forEach(out);

Answer 4

Using API18+ wilddev´s answer can be slightly improved as you don´t need Type.Builder objects. Do all this stuff in the onCreate() method:

aIn = Allocation.createSized(rs, Element.U8(rs), H*W*3/2); // what the f**k ?  This is 12 bit per pixel, giving the length of the camera data byte array !

bmpout = Bitmap.createBitmap(W, H, Bitmap.Config.ARGB_8888); // you will need this bitmap for output anyway

aOut = Allocation.createFromBitmap(rs, bmpout);  // tada !

// and set the script´s input
yuvToRgbIntrinsic.setInput(aIn);

// as pixel data are stored as int (one byte for red, green, blue, alpha), you first need an int[] array
int rgba[] = new int[w*h];   // the rgba[] array

Now you can continue with these lines or put them into the onPreviewFrame() method:

aIn.copyFrom(data);  // or aIn.copyFromUnchecked(data);  // which is faster and safe for camera data

yuvToRgbIntrinsic.forEach(aOut);  // execute the script

aOut.copyTo(bmpout);  // copy result from aOut to bmpout

// if you need the rgba-values, do
bmpout.getPixels(rgba, 0, W, 0, 0, W, H);

// now you may loop through the rgba[] array and extraxt the r,g,b,a values 
// and put them into a byte[] array(s), BUT this will surely have impact on the performance when doing in Java

Answer 5

Change your output type passed to the constructor of ScriptIntrinsicYubToRGB to be Element.U8_4 instead of Element.RGBA_8888. You'll need the same for the Type.Builder used to create the output Allocation.