UIImage created from MTKView results in color/opacity differences

Question

When I capture the contents of an MTKView into a UIImage, the resulting image looks qualitatively different, as shown below:

The code I use to generate the UIImage is as follows:

let kciOptions = [kCIContextWorkingColorSpace: CGColorSpace(name: CGColorSpace.sRGB)!,
                         kCIContextOutputPremultiplied: true,
                         kCIContextUseSoftwareRenderer: false] as [String : Any]
let lastDrawableDisplayed = self.currentDrawable! // needed to hold the last drawable presented to screen
drawingUIView.image = UIImage(ciImage: CIImage(mtlTexture: lastDrawableDisplayed.texture, options: kciOptions)!)

Since I don't modify the ciImage orientation (.oriented(CGImagePropertyOrientation.downMirrored)) the resulting image is upside down, as shown in the image above. I leave the mirrored orientation as is so I can point out the color differences between the two image captures.

No matter how I change the kciOptions parameters, (say, even changing the colorspace to grayscale) I'm not seeing any changes in the resulting UIImage, which appears much more dim/desaturated than the original. Does anybody have any suggestions for how I can accurately capture what I'm drawing on MTKView to an UIImage? Any suggestions would be much appreciated.

Below are my MTKView settings which may prove relevant:

let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
renderPipelineDescriptor.vertexFunction = vertexProgram
renderPipelineDescriptor.sampleCount = self.sampleCount
renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm
renderPipelineDescriptor.colorAttachments[0].isBlendingEnabled = true
renderPipelineDescriptor.colorAttachments[0].rgbBlendOperation = .add
renderPipelineDescriptor.colorAttachments[0].alphaBlendOperation = .add
renderPipelineDescriptor.colorAttachments[0].sourceRGBBlendFactor = .sourceAlpha         renderPipelineDescriptor.colorAttachments[0].destinationRGBBlendFactor = .oneMinusSourceAlpha            renderPipelineDescriptor.colorAttachments[0].sourceAlphaBlendFactor = .sourceAlpha              renderPipelineDescriptor.colorAttachments[0].destinationAlphaBlendFactor = .oneMinusSourceAlpha
self.isOpaque = false // makes MTKView bg transparent

Answer 1

Your CGColorSpace is .sRGB but your renderPipelineDescriptor's pixelFormat is .bgra8Unorm. Try changing that line to:

renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm_srgb

Answer 2

I've seen this issue in a couple of posts, but no clear answer. Here is what I've found:

For starters,

renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm

should really just be set to the MTKView's native pixel format

renderPipelineDescriptor.colorAttachments[0].pixelFormat = self.colorPixelFormat

Secondly, when I set the CIImage's options:

let kciOptions = [kCIContextWorkingColorSpace: CGColorSpace(name: CGColorSpace.sRGB)!,
                         kCIContextOutputPremultiplied: true,
                         kCIContextUseSoftwareRenderer: false] as [String : Any]

It didn't matter what I set kCIContextWorkingColorSpace to, I never saw any visual difference regardless of what I used. The property I really needed to set is called KCIImageColorSpace. So the updated kciOptions looks like:

let kciOptions = [kCIImageColorSpace: CGColorSpaceCreateDeviceRGB(),
                      kCIContextOutputPremultiplied: true,
                      kCIContextUseSoftwareRenderer: false] as [String : Any]

In a similar way of using the view's native pixel format, calling CGColorSpaceCreateDeviceRGB() creates an RGB colorspace that is specific to the device being used.

Answer 3

let context = CIContext()
let texture = metalView.currentDrawable!.texture
let cImg = CIImage(mtlTexture: texture, options: nil)!
let cgImg = context.createCGImage(cImg, from: cImg.extent)!
let uiImg = UIImage(cgImage: cgImg)