How to apply a Vignette CIFilter to a live camera feed in iOS?

Question

While trying to apply a simple vignette filter to the raw camera feed of an iPhone6, with the help of Metal and Core Image, I see a lot of lag between the frames being proce

Answer 1

Your step 2 is way too slow to support real-time rendering... and it looks like you're missing a couple of steps. For your purpose, you would typically:

Setup:

1. create a pool of CVPixelBuffer - using CVPixelBufferPoolCreate
2. create a pool of metal textures using CVMetalTextureCacheCreate

For each frame:

3. convert CMSampleBuffer > CVPixelBuffer > CIImage
4. Pass that CIImage through your filter pipeline
5. render the output image into a CVPixelBuffer from the pool created in step 1
6. use CVMetalTextureCacheCreateTextureFromImage to create a metal texture with your filtered CVPixelBuffer

If setup correctly, all these steps will make sure your image data stays on the GPU, as opposed to travelling from GPU to CPU and back to GPU for display.

The good news is all this is demoed in the AVCamPhotoFilter sample code from Apple https://developer.apple.com/library/archive/samplecode/AVCamPhotoFilter/Introduction/Intro.html#//apple_ref/doc/uid/TP40017556. In particular see the RosyCIRenderer class and its superclass FilterRenderer.