Raycast in Three.js with only a projection matrix

Question

I\'m rendering some custom layers using Three.js in a Mapbox GL JS page following this example. I\'d like to add raycasting to determine which object a user has clicked on.

Answer 1

Difficult to debug, but solved!

TL;DR: Full code in this working Fiddle.

1. I think that not having the world matrix of the camera from MapBox is not the main problem, rather the incompatibility of the coordinate spaces. Mapbox delivers a left-handed system with z pointing up. Three uses a right-handed system with y pointing up.

2. During the debugging I created a reduced copy of the raycaster setup functions to have everything under control and it paid off.

3. A cube is not the best object for debugging, It is way too symmetric. The best are asymmetric primitives or compound objects. I prefer a 3D coordinate cross to see the axes orientation straight away.

Coordinate systems

• There is no need for changing the DefaultUp in the BoxCustomLayer constructor, since the goals is to have everything aligned with the default coordinates in Three.
• You flip the y-axis in the onAdd() method, but then you also scale all objects when initializing the group. This way, the coordinates you operate in after inverting the projection in raycast() are not in meters anymore. So let's join that scaling part with this.cameraTransform. The left-handed to right-handed conversion should be done here as well. I decided to flip the z-axis and rotate 90deg along x-axis to get a right-handed system with y pointing up:

const centerLngLat = map.getCenter();
this.center = MercatorCoordinate.fromLngLat(centerLngLat, 0);
const {x, y, z} = this.center;

const s = this.center.meterInMercatorCoordinateUnits();

const scale = new THREE.Matrix4().makeScale(s, s, -s);
const rotation = new THREE.Matrix4().multiplyMatrices(
        new THREE.Matrix4().makeRotationX(-0.5 * Math.PI),
        new THREE.Matrix4().makeRotationY(Math.PI)); //optional Y rotation

this.cameraTransform = new THREE.Matrix4()
        .multiplyMatrices(scale, rotation)
        .setPosition(x, y, z);

3. Make sure to remove the scaling from the group in makeScene , in fact you don't need the group anymore.

4. No need to touch the render function.

Raycasting

Here it gets a bit tricky, basically it's what Raycaster would do, but I left out some unnecessary function calls, e.g. the camera world matrix is identity => no need to multiply with it.

1. Get the inverse of the projection matrix. It does not update when Object3D.projectionMatrix is assigned, so let's compute it manually.
2. Use it to get the camera and mouse position in 3D. This is equivalent to Vector3.unproject.
3. viewDirection is simply the normalized vector from the cameraPosition to the mousePosition.

const camInverseProjection = 
        new THREE.Matrix4().getInverse(this.camera.projectionMatrix);
const cameraPosition =
        new THREE.Vector3().applyMatrix4(camInverseProjection);
const mousePosition =
        new THREE.Vector3(mouse.x, mouse.y, 1)
        .applyMatrix4(camInverseProjection);
const viewDirection = mousePosition.clone()
        .sub(cameraPosition).normalize();

4. Setup the raycaster ray

this.raycaster.set(cameraPosition, viewDirection);

5. The rest can stay the same.

Full code

Fiddle for demonstration.

• I added mousemove to display real-time debug info with jQuery.
• The full info about the hit gets logged to the console upon click.