iOS 10 breaks custom CIFilter

问题

I have written a chromakey filter for making the backgrounds of MPEG movies transparent so that you can use a movie file for longer animations without the need for lengthy sequences of PNGs (as is commonly done for some types of iOS animations).

I am using AVPlayer, AVVideoComposition, and a custom CIFilter to render the video over a background image. The background image can be changed dynamically by the user interacting with the app.

This used to work just fine until iOS 10 came out and now it is broken.

What happens now is that the video plays, but no chroma keying is occurring and Xcode repeatedly spits out the following error:

need a swizzler so that YCC420v can be written.

Here's an image of what the CIFilter should produce:

result with custom CIFilter working (pre iOS 10)

And instead this is what it produces (since iOS 10):

result with broken CIFilter (post iOS 10)

Here's the section of my code that creates the EAGLContext and applies the custom CIFilter:

    let myEAGLContext = EAGLContext.init(API: EAGLRenderingAPI.OpenGLES2)
    //let cicontext = CIContext.init(EAGLContext: myEAGLContext, options: [kCIContextWorkingColorSpace: NSNull()])
    let cicontext = CIContext.init(EAGLContext: myEAGLContext)

    let filter = ChromaKeyFilter()
    filter.activeColor = CIColor.init(red: 0, green:1.0, blue: 0.0)
    filter.threshold = self.threshold

    //most of below comes from the "WWDC15 What's New In Core Image" slides
    let vidComp = AVVideoComposition(asset: videoAsset!,
                                     applyingCIFiltersWithHandler:
        {
            request in
            let input = request.sourceImage.imageByClampingToExtent()

            filter.inputImage = input

            let output = filter.outputImage!.imageByClampingToExtent()
            request.finishWithImage(output, context: cicontext)
            self.reloadInputViews()

    })

    let playerItem = AVPlayerItem(asset: videoAsset!)
    playerItem.videoComposition = vidComp
    self.player = AVPlayer(playerItem: playerItem)
    self.playerInitialized = true
    let layer = AVPlayerLayer(player: player)

    self.subviews.forEach { subview in
        subview.removeFromSuperview()
    }

    layer.frame = CGRect(x: 0.0, y: 0.0, width: self.frame.size.width, height: self.frame.size.height)
    self.layer.addSublayer(layer)

And here's the code for the custom CIFilter:

private class ChromaKeyFilter : CIFilter {
private var kernel: CIColorKernel!
var inputImage: CIImage?
var activeColor = CIColor(red: 0.0, green: 1.0, blue: 0.0)
var threshold: Float = 0.05

override init() {
    super.init()
    kernel = createKernel()
}

required init(coder aDecoder: NSCoder) {
    super.init(coder: aDecoder)!
    kernel = createKernel()
}

override var outputImage: CIImage? {
    if let inputImage = inputImage {
        let dod = inputImage.extent
        let args = [inputImage as AnyObject, activeColor as AnyObject, threshold as AnyObject]
        return kernel.applyWithExtent(dod, arguments: args)
    }
    return nil
}

private func createKernel() -> CIColorKernel {
    let kernelString =
        "kernel vec4 chromaKey( __sample s, __color c, float threshold ) { \n" +
            //below kernel was adapted from the GPUImage custom chromakeyfilter:
            //https://github.com/BradLarson/GPUImage/blob/master/framework/Source/GPUImageChromaKeyFilter.m#L30
            "  float maskY = 0.2989 * c.r + 0.5866 * c.g + 0.1145 * c.b;\n" +
            "  float maskCr = 0.7132 * (c.r - maskY);\n" +
            "  float maskCb = 0.5647 * (c.b - maskY);\n" +
            "  float Y = 0.2989 * s.rgb.r + 0.5866 * s.rgb.g + 0.1145 * s.rgb.b;\n" +
            "  float Cr = 0.7132 * (s.rgb.r - Y);\n" +
            "  float Cb = 0.5647 * (s.rgba.b - Y);\n" +
            "  float blendValue = smoothstep(threshold, threshold + 0.5, distance(vec2(Cr, Cb), vec2(maskCr, maskCb)));\n" +
            "  return blendValue * vec4( s.rgb, 1.0 ); \n" +
    "}"
    let kernel = CIColorKernel(string: kernelString)
    return kernel!
}

}

Anybody have some ideas about why this is only now breaking? Interestingly, it is only broken on the phone. It still works on the simulator, albeit much much slower than it used to before iOS 10 came out.

回答1:

It looks like some part (the player layer?) of the iOS10 (device) pipeline has switched to YUV.

Setting your AVPlayerLayer's pixelBufferAttributes to BGRA fixes the lack of alpha and silences the logged error:

layer.pixelBufferAttributes = [kCVPixelBufferPixelFormatTypeKey as String: NSNumber(value: kCVPixelFormatType_32BGRA)]

来源：https://stackoverflow.com/questions/39682268/ios-10-breaks-custom-cifilter

标签

swift

glsl

avfoundation

ios10

cifilter