Add support for Nodes system to work with JS code similar to GLSL and WGSL

[LeviPesin]

Description

I suggest to add support for Nodes system to work with JS code similar to GLSL and WGSL. I see multiple ways in which it can be useful:

• It can be useful for doing CPU/GPGPU calculations depending on environment. I.e. you can build a large Nodes program, which then can be executed either by JavaScript, by WebGPURenderer.compute(), or WebGLRenderer.compute() (I will do a PR adding WebGLRenderer.compute() soon).
• It can be used to replicate the removed SoftwareRenderer (a CPU-side renderer).

Solution

Add support for Nodes system to work with JS code. I can work on adding it.

Alternatives

Don't add support for Nodes system to work with JS code.

Additional context

No response

[sunag]

Yeah! And we will be able to use that to create interactive stuff in NodeEditor
#23165 (comment)

[pailhead]

Hi, I haven't been here in a minute so I'm probably out of the loop... But, why do you think that this should be part of three.js?

Composition should work with regular js, ie:

const myDAG = new MyWhatever(new MyWhatever('foo'))

Also you should be able to just unroll the DAG and get regular js/glsl that three already understands. In other words, having two node (nocode?) systems may bloat this project, when it could possibly be an (un)related project, in it's own repo.

If it's of any interest, i played with these topics a bit. I found it fairly trivial to implement an editor (although mine got really complicated :( ) and export three.js materials. It's very hard, maybe even impossible to generate a shader for three though, because it needs to be generated for a specific version range.
https://dusanbosnjak.com/test/nodes2/

It is also ( i think ) possible to remove all materials from three.js and leave just ShaderMaterial or at least it was possible up to 120 or 130 or so.
https://github.com/pailhead/three-refactor-chunk-material

So, what i'm trying to say, rather than (IMHO) bloating this project, it's actually possible to make it leaner and more specific to a set of problems. In the case of materials it's just validating GLSL. Three did a decent job formatting that GLSL with just string manipulation and regexs, it did not require the whole nodes system.

And then, i think it's possible to make the whole material system much leaner. There is no need for three to know about "Basic" or "Lambert" material, it could know just about "Material".

My reasoning is that working with nodes requires a nocode editor, otherwise it feels like bloat to me. Just my 2 cents :)

[LeviPesin]

The main benefit of the nodes system is flexibility.
You cannot really change an internal function in a GLSL chunk without rewriting it and including as another one. And it will be prone to break on every release.
With nodes system it is just a few lines of JS code (or raw GLSL if you want).
Nodes system doesn't produce code as clean as hand-written indeed... But that can be mostly fixed by inserting label( ..., name ) at places to make variables variables in GLSL.

My reasoning is that working with nodes requires a nocode editor, otherwise it feels like bloat to me.

There are such -- apart from Three.js' base editor there is the node editor.

(By the way, I think this isn't very related to this issue)

[pailhead]

Right, I was merely asking, why change internals of a chunk at all, or rather, why is there something "internal" here at all :)

If you are proposing more systems (such as nodes) to be part of some future renderer or even /core I believe that I'm on the right topic. If i'm totally misunderstanding this, then apologies.

[mrdoob]

I'm also confused about where NodeMaterial is going. Seems like it's not only about materials anymore.

I've proposed to move the node editor code out of examples: #25692 (comment)
Hopefully that should clarify things a bit.

[LeviPesin]

I'm also confused about where NodeMaterial is going. Seems like it's not only about materials anymore.

NodeMaterial is a material. But the nodes system is wider than just NodeMaterial -- for example, we can do GPGPU computations with it.

[pailhead]

I think thats fair, but I think the point here is that if you put a jet engine inside of three it could fly, but why, does it need to?

I think this allows you do do gpgpu with just JavaScript, it’s very old though:
https://github.com/zadvorsky/three.bas

Back in the day three was mostly concerned about rendering. Granted, this was never quite clear, but I think a lot of folks agreed that there is a “renderer”.

So taking this BAS, zadvorsky was able to create this tool off to the side and use it to whatever it’s supposed to do. He also shared it so others can do that too. But it doesn’t impact the threejs project at all.

My general take is that a lot of people use threejs for different things, and there are some (probably a minority) that would like threejs to be lean. For example I hated importing all the PBR shaders if all I’m using is my own simple shader material, but cest la vie :)

I think that in theory you could build any kind of a tool you want in parallel with threejs, and that it could actually yield cleaner code do to separation of concerns.

Again I apologize if there is something I don’t understand. It seems that threejs is much more about nodes today than it was in the past so im not sure if my arguments here are valid.

[sunag]

I think this allows you do do gpgpu with just JavaScript, it’s very old though:

I think the concept is a little different from three.bas. I think the Node System mission allows you to code TSL and Nodes in any backend or for any need like GPGPU computations. The same code programmed using TSL (Nodes) should work on WebGPU/WebGL and derivatives renderers. NodeMaterial uses the Node System in its base to build the shaders.

It's very hard, maybe even impossible to generate a shader for three though, because it needs to be generated for a specific version range.

In fact, this is one of the Node System solutions in relation to ShaderChunk.

I think that in theory you could build any kind of a tool you want in parallel with threejs, and that it could actually yield cleaner code do to separation of concerns.

It is certainly a possibility, I think that over time we add some things and eliminate others and this is part of the innovation and maintenance process. This makes a lot of sense to me as a No-Code Node Graph, @LeviPesin saw another possibility.

[pailhead]

In fact, this is one of the Node System solutions in relation to ShaderChunk.

Just to clarify this - it wouldn’t address this in old versions of three that still rely on chunks? At some point if chunks go away in favor of nodes you are saying they would be compatible with different renderers?

If so, I’d argue that the contract is still just a valid, or somewhat valid GLSL string. My reasoning is that there could be several different node standards or projects that could output this string. I never quite understood why three needs to know about the source of this string.

I think the shader incompatibility is just in the nature of this R versioning system. There is nothing indicating that a certain chunk changed in a certain version, nor why did it change.

Honestly I think that this issue would be present whatever the system is, because if the renderer changes how it interprets something wouldn’t a node ultimately run into the same issue as a chunk? At the end of the day, both would produce a string, and this string would cause an error?

[sunag]

@pailhead Comparing strings is a rationale for ShaderChunk not Node System, WebGPURenderer does not have ShaderChunk, there are no strings to compare there, there are no hacks, so it does not suffer from these problems.

[pailhead]

Just found an issue of mine that may be related. If i'm understanding this correctly, it's proposing the complete opposite direction of what's being proposed here.

#15537

[pailhead]

@sunag whoa, yeah I haven't worked with the WebGPURenderer at all, you're saying it doesn't work with ShaderMaterial at all? Lol, yeah in that case, ignore me, I'm really out of the loop 😊