Cropping a captured image exactly to how it looks in AVCaptureVideoPreviewLayer

Question

I have a photo app that is using AV Foundation. I have setup a preview layer using AVCaptureVideoPreviewLayer that takes up the top half of the screen. So when the user is tryin

Answer 1

@Cabus has a solution that works and you should up-vote his answer. However, I did my own version in Swift with the following:

// The image returned in initialImageData will be larger than what
//  is shown in the AVCaptureVideoPreviewLayer, so we need to crop it.
let image : UIImage = UIImage(data: initialImageData)!

let originalSize : CGSize
let visibleLayerFrame = self.previewView!.bounds // THE ACTUAL VISIBLE AREA IN THE LAYER FRAME

// Calculate the fractional size that is shown in the preview
let metaRect : CGRect = (self.videoPreviewLayer?.metadataOutputRectOfInterestForRect(visibleLayerFrame))!
if (image.imageOrientation == UIImageOrientation.Left || image.imageOrientation == UIImageOrientation.Right) {
    // For these images (which are portrait), swap the size of the
    // image, because here the output image is actually rotated
    // relative to what you see on screen.
    originalSize = CGSize(width: image.size.height, height: image.size.width)
}
else {
    originalSize = image.size
}

// metaRect is fractional, that's why we multiply here.
let cropRect : CGRect = CGRectIntegral(
        CGRect( x: metaRect.origin.x * originalSize.width,
                y: metaRect.origin.y * originalSize.height,
                width: metaRect.size.width * originalSize.width,
                height: metaRect.size.height * originalSize.height))

let finalImage : UIImage = 
    UIImage(CGImage: CGImageCreateWithImageInRect(image.CGImage, cropRect)!, 
        scale:1, 
        orientation: image.imageOrientation )

Answer 2

Have a look at AVCaptureVideoPreviewLayer s

-(CGRect)metadataOutputRectOfInterestForRect:(CGRect)layerRect

This method lets you easily convert the visible CGRect of your layer to the actual camera output.

One caveat: The physical camera is not mounted "top side up", but rather rotated 90 degrees clockwise. (So if you hold your iPhone - Home Button right, the camera is actually top side up).

Keeping this in mind, you have to convert the CGRect the above method gives you, to crop the image to exactly what is on screen.

Example:

CGRect visibleLayerFrame = THE ACTUAL VISIBLE AREA IN THE LAYER FRAME
CGRect metaRect = [self.previewView.layer metadataOutputRectOfInterestForRect:visibleLayerFrame];


CGSize originalSize = [originalImage size];

if (UIInterfaceOrientationIsPortrait(_snapInterfaceOrientation)) {
    // For portrait images, swap the size of the image, because
    // here the output image is actually rotated relative to what you see on screen.

    CGFloat temp = originalSize.width;
    originalSize.width = originalSize.height;
    originalSize.height = temp;
}


// metaRect is fractional, that's why we multiply here

CGRect cropRect;

cropRect.origin.x = metaRect.origin.x * originalSize.width;
cropRect.origin.y = metaRect.origin.y * originalSize.height;
cropRect.size.width = metaRect.size.width * originalSize.width;
cropRect.size.height = metaRect.size.height * originalSize.height;

cropRect = CGRectIntegral(cropRect);

This may be a bit confusing, but what made me really understand it is this:

Hold your device "Home Button right" -> You'll see the x - axis actually lies along the "height" of your iPhone, while the y - axis lies along the "width" of your iPhone. That's why for portrait images, you have to swap the size ;)

Answer 3

Here's @Erik Allen's answer in Swift 3:

let originalSize: CGSize
let visibleLayerFrame = self?.photoView.bounds

// Calculate the fractional size that is shown in the preview
let metaRect = (self?.videoPreviewLayer?.metadataOutputRectOfInterest(for: visibleLayerFrame ?? CGRect.zero)) ?? CGRect.zero

if (image.imageOrientation == UIImageOrientation.left || image.imageOrientation == UIImageOrientation.right) {
    // For these images (which are portrait), swap the size of the
    // image, because here the output image is actually rotated
    // relative to what you see on screen.
    originalSize = CGSize(width: image.size.height, height: image.size.width)
} else {
    originalSize = image.size
}

let cropRect: CGRect = CGRect(x: metaRect.origin.x * originalSize.width, y: metaRect.origin.y * originalSize.height, width: metaRect.size.width * originalSize.width, height: metaRect.size.height * originalSize.height).integral

if let finalCgImage = image.cgImage?.cropping(to: cropRect) {
    let finalImage = UIImage(cgImage: finalCgImage, scale: 1.0, orientation: image.imageOrientation)

    // User your image...
}