PassNode: Implement Auto-MRT.

[Mugen87]

Description

More advanced effects like AO, Bloom or Motion Blur require next to beauty and depth a normal and potentially a velocity buffer from the scene pass. It would be good for performance if normal and velocity would be generated in one pass with beauty and depth. To do that, the renderer needs to automatically configure the scene's materials so they output whatever is requested in the current render target.

Solution

In PassNode I have added the following two methods:

getTextureNormalNode() {

	if ( this._normalTextureNode === null ) {

		const normalTexture = this.renderTarget.textures[ 0 ].clone();
		normalTexture.name = 'PostProcessingNormal';
		normalTexture.isRenderTargetTexture = true;

		this.renderTarget.textures[ 1 ] = normalTexture;
		this._normalTextureNode = nodeObject( new PassTextureNode( this, normalTexture ) );

	}

	return this._normalTextureNode;

}

getTextureVelocityNode() {

	if ( this._velocityTextureNode === null ) {

		const velocityTexture = this.renderTarget.textures[ 0 ].clone();
		velocityTexture.name = 'PostProcessingVelocity';
		velocityTexture.isRenderTargetTexture = true;

		this.renderTarget.textures[ 2 ] = velocityTexture;
		this._velocityTextureNode = nodeObject( new PassTextureNode( this, velocityTexture ) );

	}

	return this._velocityTextureNode;

}

However, they only configure the render target and the texture nodes for further use in effects. It's also required to auto-configure all node materials in the scene to produce the correct output according to the render targets setting. I've seen OutputStructNode and its usage in webgpu_multiple_rendertargets but this class needs to be used internally somehow.

Alternatives

Doing post processing without MRT but this has performance implications when implementing more complex effects.

Additional context

#27808 tries to add this for WebGLRenderer but there is an unsolved material updating issue when switching render targets.

[Mugen87]

My plan is to advanced effects to WebGPURenderer (like GTAO or UnrealBloom) but it makes more sense doing that with Auto-MRT in place.

[sunag]

This is very important and something I was planning too, I'll try publish something for the next release.

[CodyJasonBennett]

Have you considered a g buffer pass or something with specialized packing for those effects? e.g. https://github.com/0beqz/realism-effects/blob/v2/src/gbuffer/shader/gbuffer_packing.glsl

I worry this can be very bandwidth hungry if generalized to an attachment per element of data since we can't have mixed precision between MRT attachments -- have to implement that ourselves with packing like the above.

[Mugen87]

I would recommend to start with a default MRT setup and get this going first. Investigating optimizations based on packing is something I would do after that.

In general, I'm afraid of potential artifacts that might occur when aggressively packing data so this needs to be investigated closely. What kind of experience did you make in that regard so far?

[CodyJasonBennett]

Agree on MRT as a baseline since that's quick and easy. I misunderstood this as an enhancement to the existing post-processing effects, so I was speaking from a different context.

I don't have much to say here for generalizing a packing scheme other than this among a few topics actively designed in pmndrs/postprocessing#419. The source I've shared has battled against different ANGLE backends with notes inline for hazards. I think this can be considered a separate issue with data packing functions as a prerequisite and #28708 for quality control.

[sunag]

The packages are very interesting, transpiling them into nodes would be great, which would be very compatible with the current approach.