Get RGB from a SurfaceView displaying Live camera

问题

I am displaying a live camera in SurfaceView using camera.startPreview();. Any idea on how I can get live RGB readings from the camera?

Thanks

回答1:

I thought I could get the data converted from the SurfaceView. But the best method to use is :

Set the camera's orientation to 90 degrees.
Set output format to NV21 (which is guranteed to be supported on all devices).
Set to turn the Flash ON.
Start preview in the SurfaceView.

List item

camera = Camera.open();
cameraParam = camera.getParameters();
cameraParam.setPreviewFormat(ImageFormat.NV21);
camera.setDisplayOrientation(90);
camera.setParameters(cameraParam);
cameraParam = camera.getParameters();
camera.setPreviewDisplay(surfaceHolder);
cameraParam.setFlashMode(Parameters.FLASH_MODE_TORCH);
camera.setParameters(cameraParam);
camera.startPreview();

Then, I call the setPreviewCallback and onPreviewFrame to get the incoming frame, and convert it to RGB pixel array. Which I can then get intensity of each color in the picture by averaging all pixels intensity by running myPixels array through a for loop, and checking Color.red(myPixels[i]) for each desired color (inside the onPreviewFrame).

camera.setPreviewCallback(new PreviewCallback() {
    @Override
    public void onPreviewFrame(byte[] data, Camera camera) {
        int frameHeight = camera.getParameters().getPreviewSize().height;
        int frameWidth = camera.getParameters().getPreviewSize().width;
        // number of pixels//transforms NV21 pixel data into RGB pixels  
        int rgb[] = new int[frameWidth * frameHeight];
        // convertion
        int[] myPixels = decodeYUV420SP(rgb, data, frameWidth, frameHeight);
    }
}

Where decodeYUV420SP is found here.

I timed this operation to take about 200ms for each frame. Is there a faster way of doing it?

回答2:

You can do similar something like below

camera.takePicture(shutterCallback, rawCallback, jpegCallback);

    jpegCallback = new PictureCallback() {
    public void onPictureTaken(byte[] data, Camera camera) {
        FileOutputStream outStream = null;
        try {
            Bitmap bitmap = BitmapFactory.decodeByteArray(data, offset, length);

         int[] pix = new int[picw * pich];
         bitmap.getPixels(pix, 0, picw, 0, 0, picw, pich);

         int R, G, B,Y;

         for (int y = 0; y < pich; y++){
         for (int x = 0; x < picw; x++)
             {
             int index = y * picw + x;
             int R = (pix[index] >> 16) & 0xff;     //bitwise shifting
             int G = (pix[index] >> 8) & 0xff;
             int B = pix[index] & 0xff;

             pix[index] = 0xff000000 | (R << 16) | (G << 8) | B;
             }}


        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
        }
     }
};

here camera.takePicture(shutterCallback, rawCallback, jpegCallback); method call on image capture time , so i think you need to do continually call this method while you camera is open .

回答3:

as requested here is a fast decoding using NDK ( less then 10 ms on fast device ):

first here is the native.h

#include <jni.h>
#ifndef native_H
#define native_H
extern "C" {
JNIEXPORT jbyteArray JNICALL Com_example_MainActivity_nativeSetIamgeFromCamera(JNIEnv* jenv, jobject obj,jbyteArray array,jint length,jint x,jint y);
};
#endif

native.cpp

    #include <stdint.h>
        #include <jni.h>
        #include <stdlib.h>
        #include <string.h>
        #include <stdio.h>
        #include <android/native_window.h>
        #include <android/native_window_jni.h>
        #include <sys/types.h>
        #include <stdio.h>
        #include <string.h>
        #include <sys/types.h>
        #include <time.h>
        #include "native.h"

void Resize_and_decodyuv(unsigned char * data,int _width,int _height, unsigned char *out, int newWidth, int newHeight);

JNIEXPORT jbyteArray JNICALL Com_example_MainActivity_nativeSetIamgeFromCamera(JNIEnv* jenv, jobject obj,jbyteArray array,jint length,jint x,jint y)
    {
    //-----jbyteArray array contain the data from the camera passed by the java function
    //-----length represent the size of jbyteArray in byte
    //-----x,y respectively resolutionx and resolutiony of the image in jbyteArray array

    unsigned char * buffImgCamera=(unsigned char *)malloc(length);

    //----- copy the buffer from java array to c/c++ char * buffImgCamera
    jenv->GetByteArrayRegion(array, 0, length, (jbyte*)buffImgCamera);

   int width=400,height=600;//screen reso of the surface(400,800 is just an example)
   unsigned char * buffOut=(unsigned char *)malloc(width*height*4);//prepare the result buffer where 4 represent R G B A(Alpha transparency channel).


    //--- to gain time i decode and resize the image to fit the surface screen in one loop
    Resize_and_decodyuv(buffImgCamera,x,y,buffOut,width,height);
    //---copy the result to a jbytearray and return it to java function
    jbyteArray result=env->NewByteArray(width*height*4);
    env->SetByteArrayRegion( result, 0, width*height*4, buffOut);

    return result;
    }



void Resize_and_decodyuv(unsigned char * data,int _width,int _height, unsigned char *out, int newWidth, int newHeight)
    {


int Colordeep=4;//RGBA; in the case of Qt Frame Work or Borland just put 3and the code should work;

        float scaleWidth =  (float)newWidth / (float)_width;
        float scaleHeight = (float)newHeight / (float)_height;


        for(int cy = 0; cy < newHeight; cy++)
        {
            for(int cx = 0; cx < newWidth; cx++)
            {
                int pixel = (cy * (newWidth *Colordeep)) + (cx*Colordeep);
                int nearestMatch =  ((((int)(cy / scaleHeight)) *_width) + (int)(cx /scaleWidth));

                int cxa=cx/scaleWidth;
                int cya=cy/scaleHeight; cya/=2;
                int nearestMatch1 =(cya *_width) + (int)(cxa);




                int y =  ( data[nearestMatch]);

                      int v = data[data_uv+(nearestMatch1)];
                      int u = data[data_uv+(nearestMatch1)+1];



                      int r = (int) (1164 * (y - 16) + 1596 * (v - 128));
                      int g = (int) (1164 * (y - 16) - 813 * (v - 128) - 391 * (u - 128));
                      int b = (int) (1164 * (y - 16) + 2018 * (u - 128));
          r/=1000;
          g/=1000;
          b/=1000;
                      r = r < 0 ? 0 : (r > 255 ? 255 : r);
                      g = g < 0 ? 0 : (g > 255 ? 255 : g);
                      b = b < 0 ? 0 : (b > 255 ? 255 : b);



                out[pixel ] = r;
                out[pixel  +1  ] =  g;
                out[pixel + 2] =  b;
                if(Colordeep==4)out[pixel + 3] =  255;


            }
        }


    }

java code

PreviewCallback previewCallback = new PreviewCallback () {


            public void onPreviewFrame(byte[] data, Camera camera) {
                //nativeSetIamgeFromCamera return a byte array  
                nativeSetIamgeFromCamera(data,data.length,camera.getParameters().getPreviewSize().width,camera.getParameters().getPreviewSize().height);


            }
    };

来源：https://stackoverflow.com/questions/15918180/get-rgb-from-a-surfaceview-displaying-live-camera

标签

android

colors

camera

rgb

surfaceholder