Support shadow mapping with reverse Z

[prideout]

Description

To repro, modify examples/webgl_shadowmap.html by passing reverseDepthBuffer: true into the WebGLRenderer constructor.

To get anything to render at all, you will also need to add renderer.getContext().clearDepth(0) (see my comment in #29579).

After these modifications, the demo looks okay except that there are no shadows.

Solution

Ideally shadow mapping is supported without requiring clients to write any extra code, since DepthBuffer in WebGLState already flips the depth function & depth clear value at a low level. I have not yet been able to find the missing piece.

Alternatives

Alternatively, maybe the onus is on the applications to handle this in the onBeforeShadow hook?

Additional context

No response

[eXponenta]

Another problem with shadow mapping - needs to flip compare function inside shader programs.
Because if you use strict bias checking by lower depth in fragment it will miss die to buffer inversion.

reverseZ pretty unstable because required to change everything that use depthTexture and compares depth manually.

[eXponenta]

@CodyJasonBennett
@Mugen87
Looks like we should flip this line when compare samples from depth buffer and current texel, didn't we?

three.js/src/renderers/shaders/ShaderChunk/shadowmap_pars_fragment.glsl.js

Line 78 in 09c38ab

return step( compare, unpackRGBAToDepth( texture2D( depths, uv ) ) );

[CodyJasonBennett]

Yes, the depth range is inverted. Any NDC math will also need to be adjusted for the [0, 1] range like it would for WebGPU.

[WestLangley]

Also of interest is how ShadowMapViewer should render when reverse Z is true. Perhaps the viewer should be designed to render the same, regardless. 🤔

[ShaydeNZ]

Just to confirm what @prideout described above, I have a project that would not render anything when reverseDepthBuffer was true until I added renderer.getContext().clearDepth(0) prior to rendering.

This could be a problem for anyone that tries to use reverseDepthBuffer as, if their render produces nothing, it's not obvious how to fix it. I only found the answer after coming across this report.

[CodyJasonBennett]

I don't see any NDC math that needs updating in shadowmap chunks (just the comparison), but I do observe near-field objects not receiving shadowing. Reverse depth as implemented only updates the projection matrix for the primary camera so it's both reversed and in the [0-1] clipping range #29445. Shadow cameras do receive the correct transform when reverse depth is enabled ([-1, 1] -> [0, 1] -> [1, 0]), so I expect depth packing chunks need to be updated.

[CodyJasonBennett]

Looking at ShadowMapViewer (uses 1.0 - unpackRGBAToDepth( .. ), the packed range is wrong still. It is expected for depth to still be in [-1, 1], but it is in [0, 1], appearing brighter when inverted (or darker as pictured). You can get identical results by inverting the depth value and remapping the range to [-1, 1].

Default	Reverse Depth

One hack that has shadows and ShadowMapViewer work as before is to undo reverse depth transforms when calling unpackRGBAToDepth.

Default	Reverse Depth

float unpackRGBAToDepth( const in vec4 v ) {

	float depth = dot( v, UnpackFactors4 );

	#ifdef USE_REVERSEDEPTHBUF

		// [0, 1] -> [-1, 1]
		depth = depth * 2.0 - 1.0;

		// Handle special values (background/near)
		if ( depth == 1.0 ) depth = 0.0;
		else if ( depth == 0.0 ) depth = 1.0;

		// Reverse to forward depth (precision is already destroyed at this point)
		depth = 1.0 - depth;

	#endif

	return depth;

}

The problem with this is that remapping back into [-1, 1] destroys depth precision, as that is contained between [0, 0.8]; hence, why [0, 1] NDC range is required and why it is flipped for far-field precision. It is also not exhaustive, as other methods exist that rely on non-linear depth quantities for depth moments (e.g., VSM).

Ideally, the comparison would be flipped in texture2DCompare (as eXponenta pointed out), and spot treatment would be needed for UnpackDepthRGBAShader or anything relying on a specific NDC range. Similar was needed to migrate to WebGPU since it and reverse depth rely on [0, 1] NDC range.

[WestLangley]

Taking a cursory look at this, it appears WebGLRenderer is recomputing the reversed-depth projection matrix every frame, and pushing the recomputed projection matrix to the GPU.

That seems like the wrong place, as camera.projectionMatrix (and its inverse) never honor the reversed-depth flag.

Computing the correct projection matrix once in makePerspective() and makeOrthographic() seems advisable.

That way, wherever the projection matrix is used, it will be the correct one -- and that would include shadow cameras.

/ping @Mugen87

[CodyJasonBennett]

That is a shame because we can't hide the complexity, but it might be necessary. The good news is that there is only one way to implement reverse depth, and that is with a [0, 1] clipping range. So, it could have its own coordinate system constant that is implemented the same between WebGL/WebGPU, and I have already golfed the math down so it can fit inline in makePerspective/makeOrthographic. However, reverseDepthBuffer must be configured on the renderer for it to display correctly. The only problem there is that this feature relies on EXT_clip_control in WebGL, so we would want the example displaying feature detection, regardless of renderer.

const renderer = new THREE.WebGLRenderer( { reverseDepthBuffer: true } );

const camera = new THREE.PerspectiveCamera();

if ( renderer.capabilities.reverseDepthBuffer ) { // Firefox, namely

	camera.coordinateSystem = THREE.ReversedCoordinateSystem;

}

I am not happy with this API requirement, although it would be nice if matrices could compute a reversed projection matrix.

Regarding issues with shadows, I just tracked this down to LightShadow, where the shadow projection matrix is not transformed for reverse depth as it is not attached to a camera used for rendering. This has the value later used for compare in the untransformed range, which is incorrect. It is error-prone to require users to configure the shadow camera's coordinate space, so I would have renderers update it to match the main camera. Alternatively, we can perform the transforms in WebGLShadowMap like we do the main camera.

three.js/src/lights/LightShadow.js

Lines 189 to 218 in 0201031

	/**
	* Update the matrices for the camera and shadow, used internally by the renderer.
	*
	* @param {Light} light - The light for which the shadow is being rendered.
	*/
	updateMatrices( light ) {
	const shadowCamera = this.camera;
	const shadowMatrix = this.matrix;
	_lightPositionWorld.setFromMatrixPosition( light.matrixWorld );
	shadowCamera.position.copy( _lightPositionWorld );
	_lookTarget.setFromMatrixPosition( light.target.matrixWorld );
	shadowCamera.lookAt( _lookTarget );
	shadowCamera.updateMatrixWorld();
	_projScreenMatrix.multiplyMatrices( shadowCamera.projectionMatrix, shadowCamera.matrixWorldInverse );
	this._frustum.setFromProjectionMatrix( _projScreenMatrix );
	shadowMatrix.set(
	0.5, 0.0, 0.0, 0.5,
	0.0, 0.5, 0.0, 0.5,
	0.0, 0.0, 0.5, 0.5,
	0.0, 0.0, 0.0, 1.0
	);
	shadowMatrix.multiply( _projScreenMatrix );
	}

The clear value for the shadow map looks wrong, but with special handling for that and swapping the order for comparisons in texture2DCompare, shadows work correctly. Of course, we want to ensure the clear value is correct here in the final implementation since this workaround is unexpected.

float texture2DCompare( sampler2D depths, vec2 uv, float compare ) {

	float depth = unpackRGBAToDepth( texture2D( depths, uv ) );

	#ifdef USE_REVERSEDEPTHBUF

		if ( depth == 1.0 ) depth = 0.0; // wrong clear value?
		return step( depth, compare );

	#else

		return step( compare, depth );

	#endif

}

To display the same in ShadowMapViewer, the depth range must be remapped and inverted in UnpackDepthRGBAShader. I don't think we need to care about depth precision at this point if the shader is for display purposes. Aside from handling clear values again, this is along the lines of what I would propose.

float depth = unpackRGBAToDepth( texture2D( tDiffuse, vUv ) );

#ifdef USE_REVERSEDEPTHBUF

	if ( depth == 1.0 ) depth = 0.0; // wrong clear value?

	// [0, 1] -> [-1, 1]
	depth = depth * 2.0 - 1.0;

	// Reverse to forward depth (precision is already destroyed at this point)
	depth = 1.0 - depth;

#endif

gl_FragColor = vec4( vec3( 1.0 - depth ), opacity );

If that looks okay, I can open a PR and look at GTAO. If a new coordinate space is introduced, I would advise for a warning to be emitted when using reverseDepthBuffer: true without configuring the coordinate space for the camera. An alternative would be moving the reverse depth transformations to a shader chunk, but that won't work well with existing user code. I have preferred to hide this complexity in the renderer to this point.