Antialiasing a SceneKit rendering with Metal

问题

I'm new to Metal. I'm rendering a SceneKit scene with Metal using this Apple sample code. TLDR; it calls the SCNRenderer's render function and passes in a command buffer. I'm compiling for Big Sur.

It works, but it is not anti-aliased. I've tried a few ways to achieve it, as you can see in the updates below.

Without Metal, I'd just set isJitteringEnabled to true on the SCNRenderer, and I get beautiful (and slow) 96-ish-pass renderings. If I try to do this with Metal, I get weird pixel format mismatches, so I'm suspecting the two just aren't compatible.

With Metal, as far as I can tell, the simplest way to achieve antialiasing is to enable multi-sampling in the render pipeline (I know how to do that) — and use a multi sampling texture (MTLTextureType.type2DMultisample). This partial answer backs up my assumption.

And that's the problem. I don't know how to change the texture type when I get my texture from CVMetalTextureCache and CVMetalTextureCacheCreateTextureFromImage. It seems this is a limitation in Core Video's Metal support?

My full source is here

That's it. The rest of this post is more details on the stuff I tried.

(I think this might be possible using a shader. I'm open to that solution as well, but I don't know where to start. This example doesn't compile, and this example is for GSLS)

---

My pixel buffer atts look like this

        let pixelbufferAttributes = [
            kCVPixelBufferPixelFormatTypeKey : kCVPixelFormatType_32BGRA,
            kCVPixelBufferWidthKey: exportSettings.width,
            kCVPixelBufferHeightKey : exportSettings.height,
        kCVPixelBufferMetalCompatibilityKey: true] as [String: Any]

For each frame, it creates a new pixel buffer from the pool, wraps it in a Metal texture from a cache, like this

        let pixelFormat = MTLPixelFormat.bgra8Unorm_srgb
        var optionalMetalTexture: CVMetalTexture?
        err = CVMetalTextureCacheCreateTextureFromImage(
            kCFAllocatorDefault,
            metalTextureCache, // object prop
            pixelBuffer,
            nil, // texture attributes
            pixelFormat,
            exportSettings.width,
            exportSettings.height,
            0, // planeIndex
            &optionalMetalTexture)
        guard err == noErr, let metalTexture = optionalMetalTexture else {
            fatalError("Failed to create metal texture wrapper from pixel bufffer \(err)")
        }

Attempt: Change the texture descriptor

Since I'm creating my Metal texture from a CVPixelbuffer with CVMetalTextureCacheCreateTextureFromImage, I can't figure out how to set its attributes and make it multi sample.

Attempt: Try H264

Didn't change anything. Also tried changing just the alpha quality, with HEVC with alpha, but no change.

Attempt: Enable multi sampling

I was able to get my pipeline to pick up that I wanted multi sampling, but it crashes due to the texture not being set up for multisampling (more precisely a MTLTexture of type .2DMultisample (docs)

Attempt: Copy the MTLTexture created by Core Video

I tried to use a MTLBlitCommandEncoder to copy the texture I was given by Core Video into a texture I had set up with the right attributes. But it crashes telling me that the attributes don't match.

I'm starting to think there's no solution to this?

回答1:

Enabling multisampling was the right idea. The following patch shows how to enable it.

--- a/HEVC-Videos-With-Alpha-AssetWriting/HEVC-Videos-With-Alpha-AssetWriting/AppDelegate.swift
+++ b/HEVC-Videos-With-Alpha-AssetWriting/HEVC-Videos-With-Alpha-AssetWriting/AppDelegate.swift
@@ -32,6 +32,8 @@ class AppDelegate: NSObject, NSApplicationDelegate, SCNSceneRendererDelegate {
     let renderer = SCNRenderer(device: nil, options: nil)
     var lampMaterials: SCNNode!
     var metalTextureCache: CVMetalTextureCache!
+    let msaaSampleCount = 1
+    var metalMultisampledTexture: MTLTexture!
     
     // Export
     var frameCounter = 0
@@ -61,6 +63,18 @@ class AppDelegate: NSObject, NSApplicationDelegate, SCNSceneRendererDelegate {
             fatalError("Cannot create metal texture cache: \(err)")
         }
         metalTextureCache = optionalMetalTextureCache
+        
+        if (msaaSampleCount > 1) {
+            let textureDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: MTLPixelFormat.bgra8Unorm_srgb,
+                                                                             width: ExportSettings.width,
+                                                                             height: ExportSettings.height,
+                                                                             mipmapped: false)
+            textureDescriptor.usage = .renderTarget
+            textureDescriptor.storageMode = .private
+            textureDescriptor.textureType = .type2DMultisample
+            textureDescriptor.sampleCount = msaaSampleCount
+            metalMultisampledTexture = renderer.device!.makeTexture(descriptor: textureDescriptor)
+        }
     }
     
     /// Render next frame and call the frame completion handler
@@ -106,7 +120,14 @@ class AppDelegate: NSObject, NSApplicationDelegate, SCNSceneRendererDelegate {
         let renderPassDescriptor = MTLRenderPassDescriptor()
         renderPassDescriptor.colorAttachments[0].loadAction = .clear
         renderPassDescriptor.colorAttachments[0].clearColor = clearColor
-        renderPassDescriptor.colorAttachments[0].texture = CVMetalTextureGetTexture(metalTexture)
+        if (msaaSampleCount > 1) {
+            renderPassDescriptor.colorAttachments[0].texture = metalMultisampledTexture
+            renderPassDescriptor.colorAttachments[0].resolveTexture = CVMetalTextureGetTexture(metalTexture)
+            renderPassDescriptor.colorAttachments[0].storeAction = .multisampleResolve
+        }
+        else {
+            renderPassDescriptor.colorAttachments[0].texture = CVMetalTextureGetTexture(metalTexture)
+        }
         renderer.render(atTime: currentPresentationTime.seconds,
                         viewport: ExportSettings.viewport,
                         commandBuffer: commandBuffer,

来源：https://stackoverflow.com/questions/64958392/antialiasing-a-scenekit-rendering-with-metal