SceneKit shadow on a transparent SCNFloor()

Question

I have a floor node, on which I need to cast shadow from directional light. This node needs to be transparent (used in AR environment)

Answer 1

It's been awhile since this was posted but maybe someone will find this alternative solution useful. I encountered a similar situation and what I ended up doing was rendering using multiple passes via SCNTechnique. First I rendered a floor with a solid white diffuse and then I rendered the rest of the scene without the floor. To do this I set the categoryBitMask of my SCNFloor to 3 and left the others with the default value of 1.

Next I created my SCNTechnique with this definition which renders the floor and the rest of the scene into separate buffers and then combines them together into the final scene:

self.sceneView.technique = SCNTechnique(dictionary: [
  "passes" : [
    "store-floor": [
      "draw" : "DRAW_NODE",
      "node" : "floor-node",
      "inputs" : [],
      "outputs" : [ "color" : "color_floor" ]
    ],
    "store-scene": [
      "draw" : "DRAW_SCENE",
      "excludeCategoryMask" : 2,
      "inputs" : [],
      "outputs" : [ "color" : "color_scene" ]
    ],
    "recall-scene": [
      "draw" : "DRAW_QUAD",
      "metalVertexShader" : "vertex_tecnique_basic",
      "metalFragmentShader" : "fragment_tecnique_merge",
      "inputs" : [ "alphaTex" : "color_floor", "sceneTex" : "color_scene" ],
      "outputs" : [ "color" : "COLOR" ]
    ]
  ],
  "symbols" : [
    "vertexSymbol" : [ "semantic" : "vertex" ]
  ],
  "targets" : [
    "color_floor" : [ "type" : "color" ],
    "color_scene" : [ "type" : "color" ],
  ],
  "sequence" : [
    "store-floor",
    "store-scene",
    "recall-scene"
  ]
])

Next the Metal share code that takes those two buffers and combines them together where the alpha value ranges from 0 for white to 1 for black.

using namespace metal;
#include <SceneKit/scn_metal>

struct TechniqueVertexIn
{
    float4 position [[attribute(SCNVertexSemanticPosition)]];
};

struct TechniqueVertexOut
{
  float4 framePos [[position]];
  float2 centeredLoc;
};

constexpr sampler s = sampler(coord::normalized, address::repeat, filter::linear);

vertex TechniqueVertexOut vertex_tecnique_basic(
  TechniqueVertexIn     in [[stage_in]],
  constant SCNSceneBuffer&  scnFrame [[buffer(0)]])
{
  TechniqueVertexOut vert;

  vert.framePos = float4(in.position.x, in.position.y, 0.0, 1.0);
  vert.centeredLoc = float2((in.position.x + 1.0) * 0.5 , (in.position.y + 1.0) * -0.5);

  return vert;
}

fragment half4 fragment_tecnique_merge(
  TechniqueVertexOut  vert [[stage_in]],
  texture2d<float>  alphaTex [[texture(0)]],
  texture2d<float>  sceneTex [[texture(1)]])
{
    float4 alphaColor = alphaTex.sample(s, vert.centeredLoc);
    float4 sceneColor = sceneTex.sample(s, vert.centeredLoc);
  float alpha     = 1.0 - max(max(alphaColor.r, alphaColor.g), alphaColor.b); // since floor should be white, could just pick a chan

  alpha *= alphaColor.a;
  alpha = max(sceneColor.a, alpha);

  return half4(half3(sceneColor.rgb * alpha), alpha);
}

Lastly here's an example of what that ends up looking like with all the pieces put together.

Answer 2

There are two steps to get a transparent shadow :

First : You need to connect it as a node to the scene, not as a geometry type.

let floor = SCNNode()
floor.geometry = SCNFloor()
floor.geometry?.firstMaterial!.colorBufferWriteMask = []
floor.geometry?.firstMaterial!.readsFromDepthBuffer = true
floor.geometry?.firstMaterial!.writesToDepthBuffer = true
floor.geometry?.firstMaterial!.lightingModel = .constant
scene.rootNode.addChildNode(floor)

Shadow on invisible SCNFloor():

Shadow on visible SCNPlane() and our camera is under SCNFloor():

For getting a transparent shadow you need to set a shadow color, not the object's transparency itself.

Second : A shadow color must be set like this for macOS:

lightNode.light!.shadowColor = NSColor(calibratedRed: 0,
                                               green: 0, 
                                                blue: 0, 
                                               alpha: 0.5)

...and for iOS it looks like this:

lightNode.light!.shadowColor = UIColor(white: 0, alpha: 0.5)

Alpha component here (alpha: 0.5) is an opacity of the shadow and RGB components (white: 0) is black color of the shadow.

P.S.

sceneView.backgroundColor switching between .clear colour and .white colour.

In this particular case I can't catch a robust shadow when sceneView.backgroundColor = .clear, because you need to switch between RGBA=1,1,1,1 (white mode: white colour, alpha=1) and RGBA=0,0,0,0 (clear mode: black colour, alpha=0).

In order to see semi-transparent shadow on a background the components should be RGB=1,1,1 and A=0.5, but these values are whitening the image due to internal compositing mechanism of SceneKit. But when I set RGB=1,1,1 and A=0.02 the shadow is very feeble.

Here's a tolerable workaround for now (look for solution below in SOLUTION section):

@objc func toggleTransparent() {
    transparent = !transparent
}  
var transparent = false {
    didSet {
        // this shadow is very FEEBLE and it's whitening BG image a little bit
        sceneView.backgroundColor = transparent ? 
                                    UIColor(white: 1, alpha: 0.02) : 
                                    .white
    }
}

let light = SCNLight()
light.type = .directional

if transparent == false {
    light.shadowColor = UIColor(white: 0, alpha: 0.9)
}

If I set light.shadowColor = UIColor(white: 0, alpha: 1) I'll get satisfactory shadow on BG image but solid black shadow on white.

SOLUTION:

You should grab a render of 3D objects to have premultiplied RGBA image with its useful Alpha channel. After that, you can composite rgba image of cube and its shadow over image of nature using classical OVER compositing operation in another View.

Here's a formula for OVER operation :

(RGB1 * A1) + (RGB2 * (1 – A1))