Optimizing iPhone photo-taking speed, UIImagePickerController

Question

I\'m using the UIImagePickerController in my app. Has anyone used any optimization tricks for picture-taking latency? I don\'t need to store them to the library. I simply want t

Answer 1

Are you wedded to the UIImagePickerController? As of iOS 4, AVFoundation allows you to receive a live stream of images from the camera at any supported video resolution, with no prescribed user interface, so there's no lens effect and on an iPhone 4 you can grab an image up to 720p with no latency; on earlier devices you can grab a 480p.

Session 409 of the WWDC 2010 videos available from here is a good place to start. You'll want to create an AVCaptureSession, attach a suitable AVCaptureDevice via an AVCaptureDeviceInput, add an AVCaptureVideoDataOutput and give that a dispatch queue on which to pipe data to you. You'll end up with a CVImageBufferRef, which directly exposes the raw pixel data.

EDIT: Apple's example code seemingly being missing, I tend to use approximately the following:

AVCaptureSession *session;
AVCaptureDevice *device;
AVCaptureVideoDataOutput *output;

// create a capture session
session = [[AVCaptureSession alloc] init];
session.sessionPreset = ...frame quality you want...;

// grab the default video device (which will be the back camera on a device
// with two), create an input for it to the capture session
NSError *error = nil;
device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
AVCaptureDeviceInput *input = [AVCaptureDeviceInput 
                                    deviceInputWithDevice:device error:&error];

// connect the two
[session addInput:input];

// create an object to route output back here
output = [[AVCaptureVideoDataOutput alloc] init];
[session addOutput:output];

// create a suitable dispatch queue, GCD style, and hook 
// self up as the delegate
dispatch_queue_t queue = dispatch_queue_create(NULL, NULL);
[output setSampleBufferDelegate:self queue:queue];
dispatch_release(queue);

// set 32bpp BGRA pixel format
output.videoSettings =
        [NSDictionary 
            dictionaryWithObject:
                             [NSNumber numberWithInt:kCVPixelFormatType_32BGRA]
            forKey:(id)kCVPixelBufferPixelFormatTypeKey];

[session startRunning];

That'll then start delivering CMSampleBuffers to your captureOutput:didOutputSampleBuffer:fromConnection: on the dispatch queue you created (ie, a separate thread). Obviously production code would have a lot more sanity and result checks than the above.

The following example code takes an incoming CMSampleBuffer that contains a video frame and converts it into a CGImage, then sends that off to the main thread where, in my test code, it's converted into a UIImage and set as the thing inside a UIImageView, proving that the whole thing is working:

CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

// lock momentarily, to get enough details to create a CGImage in the future...
CVPixelBufferLockBaseAddress(imageBuffer, 0);
    void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
    size_t width = CVPixelBufferGetWidth(imageBuffer);
    size_t height = CVPixelBufferGetHeight(imageBuffer);

CVPixelBufferUnlockBaseAddress(imageBuffer, 0);

// create a CGImageRef
CGColorSpaceRef colourSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef contextRef = 
    CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colourSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
CGImageRef imageRef = CGBitmapContextCreateImage(contextRef);
CGContextRelease(contextRef);
CGColorSpaceRelease(colourSpace);

[self performSelectorOnMainThread:@selector(postCGImage:) withObject:[NSValue valueWithPointer:imageRef] waitUntilDone:YES];
CGImageRelease(imageRef);

I've conflated the object that I use normally to receive video frames and some stuff from a view controller for the sake of example; hopefully I haven't made any errors.