How to use texture2d_array array in metal?

Question

I have been trying to use texture2d_array for my application of live filters in metal. But I'm not getting the proper result.

Im creating the texture array like this,

Code: Class MetalTextureArray.

class MetalTextureArray {
private(set) var arrayTexture: MTLTexture
private var width: Int
private var height: Int

init(_ width: Int, _ height: Int, _ arrayLength: Int, _ device: MTLDevice) {
    self.width = width
    self.height = height

    let textureDescriptor = MTLTextureDescriptor()

    textureDescriptor.textureType = .type2DArray
    textureDescriptor.pixelFormat = .bgra8Unorm
    textureDescriptor.width = width
    textureDescriptor.height = height
    textureDescriptor.arrayLength = arrayLength

    arrayTexture = device.makeTexture(descriptor: textureDescriptor)
}

func append(_ texture: MTLTexture) -> Bool {
    if let bytes = texture.buffer?.contents() {
        let region = MTLRegion(origin: MTLOrigin(x: 0, y: 0, z: 0), size: MTLSize(width: width, height: height, depth: 1))

        arrayTexture.replace(region: region, mipmapLevel: 0, withBytes: bytes, bytesPerRow: texture.bufferBytesPerRow)

        return true
    }

    return false
}

}

Im encoding this texture into the renderEncoder like this,

Code:

let textureArray = MetalTextureArray.init(firstTexture!.width, firstTexture!.height, colorTextures.count, device)

        _ = textureArray.append(colorTextures[0].texture)
        _ = textureArray.append(colorTextures[1].texture)
        _ = textureArray.append(colorTextures[2].texture)
        _ = textureArray.append(colorTextures[3].texture)
        _ = textureArray.append(colorTextures[4].texture)

        renderEncoder.setFragmentTexture(textureArray.arrayTexture, at: 1)

Finally I'm accessing the texture2d_array in the fragment shader like this,

Code:

struct RasterizerData {
    float4 clipSpacePosition [[position]];
    float2 textureCoordinate;

};

    multipleShader(RasterizerData in [[stage_in]],
                texture2d<half> colorTexture [[ texture(0) ]],
                texture2d_array<half> texture2D [[ texture(1) ]])
{
    constexpr sampler textureSampler (mag_filter::linear,
                                      min_filter::linear,
                                      s_address::repeat,
                                      t_address::repeat,
                                      r_address::repeat);

    // Sample the texture and return the color to colorSample

    half4 colorSample = colorTexture.sample (textureSampler, in.textureCoordinate);

    float4 outputColor;

    half red = texture2D.sample(textureSampler, in.textureCoordinate, 2).r;
    half green = texture2D.sample(textureSampler, in.textureCoordinate, 2).g;
    half blue = texture2D.sample(textureSampler, in.textureCoordinate, 2).b;

    outputColor = float4(colorSample.r * red, colorSample.g * green, colorSample.b * blue, colorSample.a);

    // We return the color of the texture
    return outputColor;
}

The textures I'm appending to the texture array are the texture which are extracted from acv curve file which is of size 256 * 1.

In this code half red = texture2D.sample(textureSampler, in.textureCoordinate, 2).r; I gave the last argument as 2 because I thought it as the index of the texture to be accessed. But I don't know what it means.

But after doing all these I'm getting the black screen. Even I have other fragment shaders and all of them are working fine. But for this fragment shader I'm getting black screen. I think for this code half blue = texture2D.sample(textureSampler, in.textureCoordinate, 2).b I'm getting 0 for all the red, green, and blue values.

Edit 1:

As suggested I used blitcommandEncoder to copy the texture and still no result.

My code goes here,

My MetalTextureArray class has come modifications.

Method append goes like this.

  func append(_ texture: MTLTexture) -> Bool {
    self.blitCommandEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0), sourceSize: MTLSize(width: texture.width, height: texture.height, depth: 1), to: self.arrayTexture, destinationSlice: count, destinationLevel: 0, destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0))
    count += 1
    return true
}

And Im appending the texture like this

let textureArray = MetalTextureArray.init(256, 1, colorTextures.count, device, blitCommandEncoder: blitcommandEncoder)
for (index, filter) in colorTextures!.enumerated() {
     _ = textureArray.append(colorTextures[index].texture)
}
renderEncoder.setFragmentTexture(textureArray.arrayTexture, at: 1)

My shader code goes like this

    multipleShader(RasterizerData in [[stage_in]],
                texture2d<half> colorTexture [[ texture(0) ]],
                texture2d_array<float> textureArray [[texture(1)]],
                const device struct SliceDataSource &sliceData [[ buffer(2) ]])
{
    constexpr sampler textureSampler (mag_filter::linear,
                                      min_filter::linear);

    // Sample the texture and return the color to colorSample
    half4 colorSample = colorTexture.sample (textureSampler, in.textureCoordinate);

    float4 outputColor = float4(0,0,0,0);

    int slice = 1;


float red = textureArray.sample(textureSampler, in.textureCoordinate, slice).r;
float blue = textureArray.sample(textureSampler, in.textureCoordinate, slice).b;
float green = textureArray.sample(textureSampler, in.textureCoordinate, slice).g;

outputColor = float4(colorSample.r * red, colorSample.g * green, colorSample.b * blue, colorSample.a);

    // We return the color of the texture
    return outputColor;
}

Still I get the black screen.

In the method textureArray.sample(textureSampler, in.textureCoordinate, slice); what is the third parameter. I though it as an index and I gave some random index to fetch the random texture. Is it correct?

Edit 2:

I finally able to implement the suggestion and I got the result by using endEncoding method before another encoder is implemented and I got the following screen with the ACV negative filter.

.

Can someone suggest me.

Thanks.

Answer 1

There's a difference between an array of textures and an array texture. It sounds to me like you just want an array of textures. In that case, you should not use texture2d_array; you should use array<texture2d<half>, 5> texture_array [[texture(1)]].

In the app code, you can either use multiple calls to setFragmentTexture() to assign textures to sequential indexes or you can use setFragmentTextures() to assign a bunch of textures to a range of indexes all at once.

In the shader code, you'd use array subscripting syntax to refer to the individual textures in the array (e.g. texture_array[2]).

If you really do want to use an array texture, then you probably need to change your append() method. First, if the texture argument was not created with the makeTexture(descriptor:offset:bytesPerRow:) method of MTLBuffer, then texture.buffer will always be nil. That is, textures only have associated buffers if they were originally created from a buffer. To copy from texture to texture, you should use a blit command encoder and its copy(from:sourceSlice:sourceLevel:sourceOrigin:sourceSize:to:destinationSlice:destinationLevel:destinationOrigin:) method.

Second, if you want to replace the texture data for a specific slice (array element) of the array texture, you need to pass that slice index in as an argument to the replace() method. For that, you'd need to use the replace(region:mipmapLevel:slice:withBytes:bytesPerRow:bytesPerImage:) method, not the replace(region:mipmapLevel:withBytes:bytesPerRow:) as you're currently doing. Your current code is just replacing the first slice over and over (assuming the source textures really are associated with a buffer).