Convert UIImage to CVPixelBufferRef

Question

I want to convert a UIImage object to a CVPixelBufferRef object, but I have absolutly no idea. And I can\'t find any example code doing anything like this.

Can someo

Answer 1

Google is always your friend. Searching for "CVPixelBufferRef" the first result leads to this snippet from snipplr:

- (CVPixelBufferRef)fastImageFromNSImage:(NSImage *)image{
CVPixelBufferRef buffer = NULL;

// config
size_t width = [image size].width;
size_t height = [image size].height;
size_t bitsPerComponent = 8; // *not* CGImageGetBitsPerComponent(image);
CGColorSpaceRef cs = CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB);
CGBitmapInfo bi = kCGImageAlphaNoneSkipFirst; // *not* CGImageGetBitmapInfo(image);
NSDictionary *d = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey, [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey, nil];

// create pixel buffer
CVPixelBufferCreate(kCFAllocatorDefault, width, height, k32ARGBPixelFormat, (CFDictionaryRef)d, &buffer);
CVPixelBufferLockBaseAddress(buffer, 0);
void *rasterData = CVPixelBufferGetBaseAddress(buffer);
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(buffer);

// context to draw in, set to pixel buffer's address
CGContextRef ctxt = CGBitmapContextCreate(rasterData, width, height, bitsPerComponent, bytesPerRow, cs, bi);
if(ctxt == NULL){
    NSLog(@"could not create context");
    return NULL;
}

// draw
NSGraphicsContext *nsctxt = [NSGraphicsContext graphicsContextWithGraphicsPort:ctxt flipped:NO];
[NSGraphicsContext saveGraphicsState];
[NSGraphicsContext setCurrentContext:nsctxt];
[image compositeToPoint:NSMakePoint(0.0, 0.0) operation:NSCompositeCopy];
[NSGraphicsContext restoreGraphicsState];

CVPixelBufferUnlockBaseAddress(buffer, 0);
CFRelease(ctxt);

return buffer;
}

No idea if this works at all, though. (Your mileage may wary :)

Answer 2

You can use Core Image to create a CVPixelBuffer from a UIImage.

// 1. Create a CIImage with the underlying CGImage encapsulated by the UIImage (referred to as 'image'):

CIImage *inputImage = [CIImage imageWithCGImage:image.CGImage];

// 2. Create a CIContext:

Context *ciContext = [CIContext contextWithCGContext:UIGraphicsGetCurrentContext() options:nil];

// 3. Render the CIImage to a CVPixelBuffer (referred to as 'outputBuffer'):

[self.ciContext render:img toCVPixelBuffer:outputBuffer];

AVFoundation provides classes that read video files (called assets) and from the output of other AVFoundation objects that handle (or have already read) assets into pixel buffers. If that is your only concern, you'll find what you're looking for in the Sample Photo Editing Extension sample code.

If your source is generated from a series of UIImage objects (perhaps there was no source file, and you are creating a new file from user-generated content), then the sample code provided above will suffice.

NOTE: It is not the most efficient means nor the only means to convert a UIImage into a CVPixelBuffer; but, it is BY FAR the easiest means. Using Core Graphics to convert a UIImage into a CVPixelBuffer requires a lot more code to set up attributes, such as pixel buffer size and colorspace, which Core Image takes care of for you.

Answer 3

A CVPixelBufferRef is what core video uses for camera input.

You can create similar pixel bitmaps from images using CGBitmapContextCreate and then drawing the image into the bitmap context.

Answer 4

There are different ways to do that, those functions convert a pixel buffer from a CGImage. UImage is a wrapper around CGImage, thus to get a CGImage you just need to call the method .CGImage.
The other ways are also create a CIImage from the buffer (already posted) or use the Accelerate framework, that is probably the fastest but also the hardest.

- (CVPixelBufferRef) pixelBufferFromCGImage: (CGImageRef) image
{
    NSDictionary *options = @{
                              (NSString*)kCVPixelBufferCGImageCompatibilityKey : @YES,
                              (NSString*)kCVPixelBufferCGBitmapContextCompatibilityKey : @YES,
                              };

    CVPixelBufferRef pxbuffer = NULL;
    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, CGImageGetWidth(image),
                        CGImageGetHeight(image), kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options,
                        &pxbuffer);
    if (status!=kCVReturnSuccess) {
        NSLog(@"Operation failed");
    }
    NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);

    CVPixelBufferLockBaseAddress(pxbuffer, 0);
    void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);

    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(pxdata, CGImageGetWidth(image),
                                                 CGImageGetHeight(image), 8, 4*CGImageGetWidth(image), rgbColorSpace,
                                                 kCGImageAlphaNoneSkipFirst);
    NSParameterAssert(context);

    CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
    CGAffineTransform flipVertical = CGAffineTransformMake( 1, 0, 0, -1, 0, CGImageGetHeight(image) );
    CGContextConcatCTM(context, flipVertical);
    CGAffineTransform flipHorizontal = CGAffineTransformMake( -1.0, 0.0, 0.0, 1.0, CGImageGetWidth(image), 0.0 );
    CGContextConcatCTM(context, flipHorizontal);

    CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
                                           CGImageGetHeight(image)), image);
    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);

    CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
    return pxbuffer;
}

Answer 5

Very late though but for someone who needs.

 // call like this
    CVPixelBufferRef videobuffer = [self pixelBufferFromCGImage:yourImage.CGImage]; 

// method that converts

-(CVPixelBufferRef)pixelBufferFromCGImage:(CGImageRef)image{

    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey, nil];
    CVPixelBufferRef pxbuffer = NULL;
    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, videoSize.width, videoSize.height,
                                          kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef)options, &pxbuffer);
    NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);

    CVPixelBufferLockBaseAddress(pxbuffer, 0);
    void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
    NSParameterAssert(pxdata != NULL);

    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(pxdata, videoSize.width, videoSize.height, 8, 4*videoSize.width, rgbColorSpace, kCGImageAlphaNoneSkipFirst);
    NSParameterAssert(context);
    CGContextConcatCTM(context, CGAffineTransformIdentity);
    CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image), CGImageGetHeight(image)), image);
    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);
    CVPixelBufferUnlockBaseAddress(pxbuffer, 0);

    return pxbuffer;
}