Reliable access and modify captured camera frames under SceneKit

Question

I try to add a B&W filter to the camera images of an ARSCNView, then render colored AR objects over it.

I\'am almost there with the following code added

Answer 1

Here's the key for this problem:

session:didUpdateFrame:

Provides a newly captured camera image and accompanying AR information to the delegate.

So just moved CVPixelBufferRef manipulation, the image filtering code from

- (void)renderer:(id<SCNSceneRenderer>)aRenderer updateAtTime:(NSTimeInterval)time

to

- (void)session:(ARSession *)session didUpdateFrame:(ARFrame *)frame

Made sure to set self.sceneView.session.delegate = self to have this delegate called.