I understand bitmap layout and pixel format subject pretty well, but getting an issue when working with png / jpeg images loaded through NSImage
let nsImage:NSImage = NSImage(byReferencingURL: …) let cgImage:CGImage = nsImage.CGImageForProposedRect(nil, context: nil, hints: nil)! let bitmapInfo:CGBitmapInfo = CGImageGetBitmapInfo(cgImage) Swift.print(bitmapInfo.contains(CGBitmapInfo.ByteOrderDefault)) // True
My kCGBitmapByteOrder32Host
Does anybody knows what's going on? Surely the system somehow knows how do deal with this, since pngs are displayed correctly. Is there a bullet-proof way detecting pixel format of CGImage? Complete demo project is available at GitHub.
P. S. I'm copying raw pixel data via CFDataGetBytePtr buffer into another library buffer, which is then gets processed and saved. In order to do so, I need to explicitly specify pixel format. Actual images I'm dealing with (any png / jpeg files that I've checked) display correctly, for example:
But bitmap info of the same images gives me incorrect endianness information, resulting in bitmap being handled as BGRA pixel format instead of actual RGBA, when I process it the result looks like this:
The resulting image demonstrates the colour swapping between red and blue pixels, if RGBA pixel format is specified explicitly, everything works out perfectly, but I need this detection to be automated.
P. P. S. Documentation briefly mentions that CGColorSpace is another important variable that defines pixel format / byte order, but I found no mentions how to get it out of there.
Could you use NSBitmapFormat?
I wrote a class to source color schemes from images, and that's what I used to determine the bitmap format. Here's a snippet of how I used it:
var averageColorImage: CIImage? var averageColorImageBitmap: NSBitmapImageRep //... core image filter code averageColorImage = filter?.outputImage averageColorImageBitmap = NSBitmapImageRep(CIImage: averageColorImage!) let red, green, blue: Int switch averageColorImageBitmap.bitmapFormat { case NSBitmapFormat.NSAlphaFirstBitmapFormat: red = Int(averageColorImageBitmap.bitmapData.advancedBy(1).memory) green = Int(averageColorImageBitmap.bitmapData.advancedBy(2).memory) blue = Int(averageColorImageBitmap.bitmapData.advancedBy(3).memory) default: red = Int(averageColorImageBitmap.bitmapData.memory) green = Int(averageColorImageBitmap.bitmapData.advancedBy(1).memory) blue = Int(averageColorImageBitmap.bitmapData.advancedBy(2).memory) }
Some years later and after testing my findings in production I can share them with good confidence, but hoping someone with theory knowledge will explain things better here? Good places to refresh memory:
Based on that you can use following extensions:
Note, that you should always pay attention to colour spaceCGColorSpace(name: CGColorSpace.sRGB)
To see this in practice drop above and the following into a playground. You'll need two one-pixel red images with alpha and without, this and this should work.
import AppKit import CoreGraphics extension CFData { public var pixelComponents: [UInt8] { let buffer: UnsafeMutablePointer<UInt8> = UnsafeMutablePointer.allocate(capacity: 4) defer { buffer.deallocate(capacity: 4) } CFDataGetBytes(self, CFRange(location: 0, length: CFDataGetLength(self)), buffer) return Array(UnsafeBufferPointer(start: buffer, count: 4)) } } let color: NSColor = .red Thread.sleep(forTimeInterval: 2) // Must flip coordinates to capture what we want… let screen: NSScreen = NSScreen.screens.first(where: { $0.frame.contains(NSEvent.mouseLocation) })! let rect: CGRect = CGRect(origin: CGPoint(x: NSEvent.mouseLocation.x - 10, y: screen.frame.height - NSEvent.mouseLocation.y), size: CGSize(width: 1, height: 1)) Swift.print("Will capture image with \(rect) frame.") let screenImage: CGImage = CGWindowListCreateImage(rect, [], kCGNullWindowID, [])! let urlImageWithAlpha: CGImage = NSImage(byReferencing: URL(fileURLWithPath: "/Users/ianbytchek/Downloads/red-pixel-with-alpha.png")).cgImage(forProposedRect: nil, context: nil, hints: nil)! let urlImageNoAlpha: CGImage = NSImage(byReferencing: URL(fileURLWithPath: "/Users/ianbytchek/Downloads/red-pixel-no-alpha.png")).cgImage(forProposedRect: nil, context: nil, hints: nil)! Swift.print(screenImage.colorSpace!, screenImage.bitmapInfo, screenImage.bitmapInfo.pixelFormat!, screenImage.dataProvider!.data!.pixelComponents) Swift.print(urlImageWithAlpha.colorSpace!, urlImageWithAlpha.bitmapInfo, urlImageWithAlpha.bitmapInfo.pixelFormat!, urlImageWithAlpha.dataProvider!.data!.pixelComponents) Swift.print(urlImageNoAlpha.colorSpace!, urlImageNoAlpha.bitmapInfo, urlImageNoAlpha.bitmapInfo.pixelFormat!, urlImageNoAlpha.dataProvider!.data!.pixelComponents) let formats: [CGBitmapInfo.RawValue] = [ CGImageAlphaInfo.premultipliedFirst.rawValue, CGImageAlphaInfo.noneSkipFirst.rawValue, CGImageAlphaInfo.premultipliedLast.rawValue, CGImageAlphaInfo.noneSkipLast.rawValue, ] for format in formats { // This "paints" and prints out components in the order they are stored in data. let context: CGContext = CGContext(data: nil, width: 1, height: 1, bitsPerComponent: 8, bytesPerRow: 32, space: CGColorSpace(name: CGColorSpace.sRGB)!, bitmapInfo: format)! let components: UnsafeBufferPointer<UInt8> = UnsafeBufferPointer(start: context.data!.assumingMemoryBound(to: UInt8.self), count: 4) context.setFillColor(red: 1 / 0xFF, green: 2 / 0xFF, blue: 3 / 0xFF, alpha: 1) context.fill(CGRect(x: 0, y: 0, width: 1, height: 1)) Swift.print(context.colorSpace!, context.bitmapInfo, context.bitmapInfo.pixelFormat!, Array(components)) }
This will output the following. Pay attention how screen-captured image differs from ones loaded from disk.
Will capture image with (285.7734375, 294.5, 1.0, 1.0) frame. <CGColorSpace 0x7fde4e9103e0> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; iMac) CGBitmapInfo(rawValue: 8194) bgra [27, 13, 252, 255] <CGColorSpace 0x7fde4d703b20> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; Color LCD) CGBitmapInfo(rawValue: 3) rgba [235, 73, 53, 255] <CGColorSpace 0x7fde4e915dc0> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; Color LCD) CGBitmapInfo(rawValue: 5) rgba [235, 73, 53, 255] <CGColorSpace 0x7fde4d60d390> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; sRGB IEC61966-2.1) CGBitmapInfo(rawValue: 2) argb [255, 1, 2, 3] <CGColorSpace 0x7fde4d60d390> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; sRGB IEC61966-2.1) CGBitmapInfo(rawValue: 6) argb [255, 1, 2, 3] <CGColorSpace 0x7fde4d60d390> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; sRGB IEC61966-2.1) CGBitmapInfo(rawValue: 1) rgba [1, 2, 3, 255] <CGColorSpace 0x7fde4d60d390> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; sRGB IEC61966-2.1) CGBitmapInfo(rawValue: 5) rgba [1, 2, 3, 255]
