Crop camera preview for TextureView

Question

I have a TextureView with a fixed width and height and I want to show a camera preview inside of it. I need to crop the camera preview so that it doesn\'t look stretched ins

Answer 1

you could manipulate the byte[] data from onPreview().

I think you'll have to:

• put it in an Bitmap
• do the cropping in the Bitmap
• do a little stretching/resizing
• and pass the Bitmap to your SurfaceView

This is not a very performant way. Maybe you can manipulate the byte[] directly, but you will have to deal with picture formats like NV21.

Answer 2

Just calculate the aspect ratio, generate a scaling matrix and apply it to the TextureView. Based on the aspect ratio of the surface and the aspect ratio of the preview image, the preview image is cropped on the top and the bottom or left and right. Another solution I found out is that if you open the camera before the SurfaceTexture is available, the preview is already scaled automatically. Just try to move mCamera = Camera.open(); to your onCreate function after you set the SurfaceTextureListener. This worked for me on the N4. With this solution you'll probably get problems when you rotate from portrait to landscape. If you need portrait and landscape support, then take the solution with the scale matrix!

private void initPreview(SurfaceTexture surface, int width, int height) {
    try {
        camera.setPreviewTexture(surface);
    } catch (Throwable t) {
        Log.e("CameraManager", "Exception in setPreviewTexture()", t);
    }

    Camera.Parameters parameters = camera.getParameters();
    previewSize = parameters.getSupportedPreviewSizes().get(0);

    float ratioSurface = width > height ? (float) width / height : (float) height / width;
    float ratioPreview = (float) previewSize.width / previewSize.height;

    int scaledHeight = 0;
    int scaledWidth = 0;
    float scaleX = 1f;
    float scaleY = 1f;

    boolean isPortrait = false;

    if (previewSize != null) {
        parameters.setPreviewSize(previewSize.width, previewSize.height);
        if (display.getRotation() == Surface.ROTATION_0 || display.getRotation() == Surface.ROTATION_180) {
            camera.setDisplayOrientation(display.getRotation() == Surface.ROTATION_0 ? 90 : 270);
            isPortrait = true;
        } else if (display.getRotation() == Surface.ROTATION_90 || display.getRotation() == Surface.ROTATION_270) {
            camera.setDisplayOrientation(display.getRotation() == Surface.ROTATION_90 ? 0 : 180);
            isPortrait = false;
        }
        if (isPortrait && ratioPreview > ratioSurface) {
            scaledWidth = width;
            scaledHeight = (int) (((float) previewSize.width / previewSize.height) * width);
            scaleX = 1f;
            scaleY = (float) scaledHeight / height;
        } else if (isPortrait && ratioPreview < ratioSurface) {
            scaledWidth = (int) (height / ((float) previewSize.width / previewSize.height));
            scaledHeight = height;
            scaleX = (float) scaledWidth / width;
            scaleY = 1f;
        } else if (!isPortrait && ratioPreview < ratioSurface) {
            scaledWidth = width;
            scaledHeight = (int) (width / ((float) previewSize.width / previewSize.height));
            scaleX = 1f;
            scaleY = (float) scaledHeight / height;
        } else if (!isPortrait && ratioPreview > ratioSurface) {
            scaledWidth = (int) (((float) previewSize.width / previewSize.height) * width);
            scaledHeight = height;
            scaleX = (float) scaledWidth / width;
            scaleY = 1f;
        }           
        camera.setParameters(parameters);
    }

    // calculate transformation matrix
    Matrix matrix = new Matrix();

    matrix.setScale(scaleX, scaleY);
    textureView.setTransform(matrix);
}

Answer 3

I have just made a working app where I needed to show a preview with x2 of the actual input without getting a pixelated look. Ie. I needed to show the center part of a 1280x720 live preview in a 640x360 TextureView.

Here is what I did.

Set the camera preview to x2 resolution of what I needed:

params.setPreviewSize(1280, 720);

And then scale the texture view accordingly:

this.captureView.setScaleX(2f);
this.captureView.setScaleY(2f);

This runs without any hiccups on small devices.

Answer 4

Provided earlier solution by @Romanski works fine but it scales with cropping. If you need to scale to fit, then use the following solution. Call updateTextureMatrix every time when surface view is changed: i.e. in onSurfaceTextureAvailable and in onSurfaceTextureSizeChanged methods. Also note that this solution relies that activity ignores configuration changes (i.e. android:configChanges="orientation|screenSize|keyboardHidden" or something like that):

private void updateTextureMatrix(int width, int height)
{
    boolean isPortrait = false;

    Display display = getWindowManager().getDefaultDisplay();
    if (display.getRotation() == Surface.ROTATION_0 || display.getRotation() == Surface.ROTATION_180) isPortrait = true;
    else if (display.getRotation() == Surface.ROTATION_90 || display.getRotation() == Surface.ROTATION_270) isPortrait = false;

    int previewWidth = orgPreviewWidth;
    int previewHeight = orgPreviewHeight;

    if (isPortrait)
    {
        previewWidth = orgPreviewHeight;
        previewHeight = orgPreviewWidth;
    }

    float ratioSurface = (float) width / height;
    float ratioPreview = (float) previewWidth / previewHeight;

    float scaleX;
    float scaleY;

    if (ratioSurface > ratioPreview)
    {
        scaleX = (float) height / previewHeight;
        scaleY = 1;
    }
    else
    {
        scaleX = 1;
        scaleY = (float) width / previewWidth;
    }

    Matrix matrix = new Matrix();

    matrix.setScale(scaleX, scaleY);
    textureView.setTransform(matrix);

    float scaledWidth = width * scaleX;
    float scaledHeight = height * scaleY;

    float dx = (width - scaledWidth) / 2;
    float dy = (height - scaledHeight) / 2;
    textureView.setTranslationX(dx);
    textureView.setTranslationY(dy);
}

Also you need the following fields:

private int orgPreviewWidth;
private int orgPreviewHeight;

initialize it in onSurfaceTextureAvailable mathod before calling updateTextureMatrix:

Camera.Parameters parameters = camera.getParameters();
parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_CONTINUOUS_VIDEO);

Pair<Integer, Integer> size = getMaxSize(parameters.getSupportedPreviewSizes());
parameters.setPreviewSize(size.first, size.second);

orgPreviewWidth = size.first;
orgPreviewHeight = size.second;

camera.setParameters(parameters);

getMaxSize method:

private static Pair<Integer, Integer> getMaxSize(List<Camera.Size> list)
{
    int width = 0;
    int height = 0;

    for (Camera.Size size : list) {
        if (size.width * size.height > width * height)
        {
            width = size.width;
            height = size.height;
        }
    }

    return new Pair<Integer, Integer>(width, height);
}

And last thing - you need to correct camera rotation. So call setCameraDisplayOrientation method in Activity onConfigurationChanged method (and also make initial call in onSurfaceTextureAvailable method):

public static void setCameraDisplayOrientation(Activity activity, int cameraId, Camera camera)
{
    Camera.CameraInfo info = new Camera.CameraInfo();
    Camera.getCameraInfo(cameraId, info);
    int rotation = activity.getWindowManager().getDefaultDisplay().getRotation();
    int degrees = 0;
    switch (rotation)
    {
        case Surface.ROTATION_0:
            degrees = 0;
            break;
        case Surface.ROTATION_90:
            degrees = 90;
            break;
        case Surface.ROTATION_180:
            degrees = 180;
            break;
        case Surface.ROTATION_270:
            degrees = 270;
            break;
    }

    int result;
    if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT)
    {
        result = (info.orientation + degrees) % 360;
        result = (360 - result) % 360;  // compensate the mirror
    }
    else
    {  // back-facing
        result = (info.orientation - degrees + 360) % 360;
    }
    camera.setDisplayOrientation(result);

    Camera.Parameters params = camera.getParameters();
    params.setRotation(result);
    camera.setParameters(params);
}

Answer 5

Answer given by @SatteliteSD is the most appropriate one. Every camera supports only certain preview sizes which is set at HAL. Hence if the available preview sizes doesn't suffice the requirement then you need to extract data from onPreview

Answer 6

For real-time manipulations of preview images of the camera, OpenCV for android is perfectly adapted. You'll find every sample needed here : http://opencv.org/platforms/android/opencv4android-samples.html, and as you'll see it works damn well in real time.

Disclaimer : setting up an opencv lib on android can be quite the headache depending on how you are experimented with c++/opencv/the ndk. In all cases, writing opencv code is never simple, but on the other hand it's a very powerful library.