Transpose z-position from perspective to orthographic camera in three.js

Question

I have a scene where I want to combine perspective objects (ie. objects that appear smaller when they are far away) with orthogographic objects (ie. objects that appear the

Answer 1

You are very close to the result what you have expected. You have forgotten to update the camera matrices, which have to be calculated that the operation project and project can proper work:

pCam.updateMatrixWorld ( false );
oCam.updateMatrixWorld ( false );
circle.position.project(pCam).unproject(oCam);

Explanation:

In a rendering, each mesh of the scene usually is transformed by the model matrix, the view matrix and the projection matrix.

• Projection matrix:
  The projection matrix describes the mapping from 3D points of a scene, to 2D points of the viewport. The projection matrix transforms from view space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) in the range (-1, -1, -1) to (1, 1, 1) by dividing with the w component of the clip coordinates.

• View matrix:
  The view matrix describes the direction and position from which the scene is looked at. The view matrix transforms from the wolrd space to the view (eye) space. In the coordinat system on the viewport, the X-axis points to the left, the Y-axis up and the Z-axis out of the view (Note in a right hand system the Z-Axis is the cross product of the X-Axis and the Y-Axis).

• Model matrix:
  The model matrix defines the location, oriantation and the relative size of a mesh in the scene. The model matrix transforms the vertex positions from of the mesh to the world space.

If a fragment is drawn "behind" or "before" another fragment, depends on the depth value of the fragment. While for orthographic projection the Z coordinate of the view space is linearly mapped to the depth value, in perspective projection it is not linear.

In general, the depth value is calculated as follows:

float ndc_depth = clip_space_pos.z / clip_space_pos.w;
float depth = (((farZ-nearZ) * ndc_depth) + nearZ + farZ) / 2.0;

The projection matrix describes the mapping from 3D points of a scene, to 2D points of the viewport. It transforms from eye space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) by dividing with the w component of the clip coordinates.

At Orthographic Projection the coordinates in the eye space are linearly mapped to normalized device coordinates.

Orthographic Projection

At Orthographic Projection the coordinates in the eye space are linearly mapped to normalized device coordinates.

Orthographic Projection Matrix:

r = right, l = left, b = bottom, t = top, n = near, f = far 

2/(r-l)         0               0               0
0               2/(t-b)         0               0
0               0               -2/(f-n)        0
-(r+l)/(r-l)    -(t+b)/(t-b)    -(f+n)/(f-n)    1

At Orthographic Projection, the Z component is calcualted by the linear function:

z_ndc = z_eye * -2/(f-n) - (f+n)/(f-n)

Perspective Projection

At Perspective Projection the projection matrix describes the mapping from 3D points in the world as they are seen from of a pinhole camera, to 2D points of the viewport.
The eye space coordinates in the camera frustum (a truncated pyramid) are mapped to a cube (the normalized device coordinates).

Perspective Projection

Perspective Projection Matrix:

r = right, l = left, b = bottom, t = top, n = near, f = far

2*n/(r-l)      0              0                0
0              2*n/(t-b)      0                0
(r+l)/(r-l)    (t+b)/(t-b)    -(f+n)/(f-n)    -1    
0              0              -2*f*n/(f-n)     0

At Perspective Projection, the Z component is calcualted by the rational function:

z_ndc = ( -z_eye * (f+n)/(f-n) - 2*f*n/(f-n) ) / -z_eye

See a detailed description at the answer to the Stack Overflow question How to render depth linearly in modern OpenGL with gl_FragCoord.z in fragment shader?

In your case this means, that you have to choose the Z coordinate of the circle in the orthographic projection in that way, that the depth value is inbetween of the depths of the objects in the perspective projection.
Since the depth value in nothing else than depth = z ndc * 0.5 + 0.5 in both cases, it also possible to do the calculations by normalized device coordinates instead of depth values.

The normalized device coordinates can easily be caluclated by the project function of the THREE.PerspectiveCamera. The project converrts from wolrd space to view space and from view space to normalized device coordinates.

To find a Z coordinate which is in between in orthographic projection, the middle normalized device Z coordinate, has to be transformed to a view space Z coordinate. This can be done by the unproject function of the THREE.PerspectiveCamera. The unproject converts from normalized device coordinates to view space and from view space to world sapce.

See further OpenGL - Mouse coordinates to Space coordinates.

See the example:

var renderer, pScene, oScene, pCam, oCam, frontPlane, backPlane, circle;

  var init = function () {
    pScene = new THREE.Scene();
    oScene = new THREE.Scene();
    
    pCam = new THREE.PerspectiveCamera(40, window.innerWidth / window.innerHeight, 1, 1000);
    pCam.position.set(0, 40, 50);
    pCam.lookAt(new THREE.Vector3(0, 0, -50));
    
    oCam = new THREE.OrthographicCamera(window.innerWidth / -2, window.innerWidth / 2, window.innerHeight / 2, window.innerHeight / -2, 1, 500);
    oCam.Position = pCam.position.clone();
    
    pScene.add(pCam);
    pScene.add(new THREE.AmbientLight(0xFFFFFF));
    
    oScene.add(oCam);
    oScene.add(new THREE.AmbientLight(0xFFFFFF));
    
    
    frontPlane = new THREE.Mesh(new THREE.PlaneGeometry(20, 20), new THREE.MeshBasicMaterial( { color: 0x990000 }));
    frontPlane.position.z = -50;
    pScene.add(frontPlane);
    
    backPlane = new THREE.Mesh(new THREE.PlaneGeometry(20, 20), new THREE.MeshBasicMaterial( { color: 0x009900 }));
    backPlane.position.z = -100;
    pScene.add(backPlane);

    circle = new THREE.Mesh(new THREE.CircleGeometry(20, 20), new THREE.MeshBasicMaterial( { color: 0x000099 }));
    circle.position.z = -75;

    
    //Transform position from perspective camera to orthogonal camera -> doesn't work, the circle is displayed in front
    pCam.updateMatrixWorld ( false );
    oCam.updateMatrixWorld ( false );
    circle.position.project(pCam).unproject(oCam);
    
    oScene.add(circle);
    
    renderer = new THREE.WebGLRenderer();
    renderer.setSize(window.innerWidth, window.innerHeight);
    document.body.appendChild(renderer.domElement);
  };
  
  var render = function () {
  
    renderer.autoClear = false;
    renderer.render(oScene, oCam);
    renderer.render(pScene, pCam);
  };
  
  var animate = function () {
      requestAnimationFrame(animate);
      //controls.update();
      render();
  };
  
  
  init();
  animate();

html,body {
    height: 100%;
    width: 100%;
    margin: 0;
    overflow: hidden;
}

<script src="https://threejs.org/build/three.min.js"></script>

Answer 2

I have found a solution that involves only the perspective camera and scales the adornments according to their distance to the camera. It is similar to the answer posted to a similar question, but not quite the same. My specific issue is that I don't only need the adornments to be the same size independent of their distance to the camera, I also need to control their exact size on screen.

To scale them to the right size, not to any size that does not change, I use the function to calculate on screen size found in this answer to calculate the position of both ends of a vector of a known on-screen length and check the length of the projection to the screen. From the difference in length I can calculate the exact scaling factor:

var widthVector = new THREE.Vector3( 100, 0, 0 );
widthVector.applyEuler(pCam.rotation);

var baseX = getScreenPosition(circle, pCam).x;
circle.position.add(widthVector);
var referenceX = getScreenPosition(circle, pCam).x;
circle.position.sub(widthVector);

var scale = 100 / (referenceX - baseX);
circle.scale.set(scale, scale, scale);

The problem with this solution is that in most of the cases the calculation is precise enough to provide an exact size. But every now and then some rounding error makes the adornment not render correctly.