Resulting MTLTexture lighter than CGImage

问题

I have kernel func which must convert Y and CbCr textures created from pixelBuffer(ARFrame.capturedImage) to RGB texture like in apple guide https://developer.apple.com/documentation/arkit/displaying_an_ar_experience_with_metal But I get over lighted texture

kernel void renderTexture(texture2d<float, access::sample> capturedImageTextureY [[ texture(0) ]],
                          texture2d<float, access::sample> capturedImageTextureCbCr [[ texture(1) ]],
                      
                          texture2d<float, access::read_write> outTextue [[texture(2)]],
                      
                      
                          uint2 size [[threads_per_grid]],
                          uint2 pid [[thread_position_in_grid]]){


constexpr sampler colorSampler(mip_filter::linear,
                               mag_filter::linear,
                               min_filter::linear);


const float4x4 ycbcrToRGBTransform = float4x4(
    float4(+1.0000f, +1.0000f, +1.0000f, +0.0000f),
    float4(+0.0000f, -0.3441f, +1.7720f, +0.0000f),
    float4(+1.4020f, -0.7141f, +0.0000f, +0.0000f),
    float4(-0.7010f, +0.5291f, -0.8860f, +1.0000f)
);

float2 texCoord;
texCoord.x = float(pid.x) / size.x;
texCoord.y = float(pid.y) / size.y;

// Sample Y and CbCr textures to get the YCbCr color at the given texture coordinate
float4 ycbcr = float4(capturedImageTextureY.sample(colorSampler,    texCoord).r,
                      capturedImageTextureCbCr.sample(colorSampler, texCoord).rg, 1.0);

float4 color = ycbcrToRGBTransform * ycbcr;

outTextue.write(color, pid);

}

I create CGImage with this code:

var cgImage: CGImage?
VTCreateCGImageFromCVPixelBuffer(pixelBuffer, options: nil, imageOut: &cgImage)

cgImage has normal lightning

when I try to create texture from cgImage with MTKTextureLoader I get over lighted texture too

How to get MTLTexture with normal light like in cgImage

cgImage: (expected result)

kernel func:

create texture with this code:

let descriptor = MTLTextureDescriptor()
descriptor.width = Int(Self.maxTextureSize.width)
descriptor.height = Int(Self.maxTextureSize.height)
descriptor.usage = [.shaderWrite, .shaderRead]
    
let texture = MTLCreateSystemDefaultDevice()?.makeTexture(descriptor: descriptor)

and write pixels with kernel func.

already tried different pixelFormats of MTLTextureDescriptor

textureLoader:

let textureLoader = MTKTextureLoader(device: MTLCreateSystemDefaultDevice()!)
let texturee = try! textureLoader.newTexture(cgImage: cgImage!, options: [.SRGB : (false as NSNumber)])

already tried different MTKTextureLoader.Options

GitHub project demonstrating issue: PixelBufferToMTLTexture

回答1:

Problem was solved Thanks 0xBFE1A8, by adding gamma correction

by replacing

outTextue.write(color, pid);

with:

outTextue.write(float4(pow(color.rgb, float3(2,2,2)), color.a), pid);

回答2:

when I try to create texture from cgImage with MTKTextureLoader I get over lighted texture

It's because metal applies gamma correction to your texture.

MTKTextureLoader has an SRGB key that is used to specify whether the texture data is stored as sRGB image data.

If the value is false, the image data is treated as linear pixel data. If the value is true, the image data is treated as sRGB pixel data. If this key is not specified and the image being loaded has been gamma-corrected, the image data uses the specified sRGB information.

let path = Bundle.main.path(forResource: "yourTexture", ofType: "png")!
let data = NSData(contentsOfFile: path) as! Data
let texture = try! textureLoader.newTexture(with: data, options: [MTKTextureLoaderOptionSRGB : (false as NSNumber)])

You can also solve this problem by adding a gamma correcting equation to your shader.