NodeMaterial

[sunag]

Hi.

I started developing of a THREE.NodeMaterial to reconcile the differences materials between 3D authoring software. In SEA3D Studio has options to create layers in Albedo with mask and various blend modes, Rim shader and others without need custom shader code. I would like to bring this to Three.JS with node shader.

I think that MeshPhongMaterial, MeshPhysicalMaterial and others can easily be based on NodeMaterial through of a interface for backward compatibility or proxy only.

UPDATED
http://sunag.github.io/sea3d/Labs/Three.JS-NodeMaterial/webgl_materials_nodes.html
http://sunag.github.io/sea3d/Labs/Three.JS-NodeMaterial/webgl_postprocessing_nodes.html

Syntax example for uses UV1 or UV2 for texture:

var usesUv2 = true;
var isLightmap = false;

var t = new THREE.NodeTexture( texture, new THREE.NodeUV( usesUv2 ) );

var nodemat = new THREE.NodePhongMaterial();
if (isLightmap) nodemat.light = t;
else nodemat.color = t;
nodemat.build(); // build shader

I am making an editor too, currently this would be the interface. Color is albedo and transform is the vertex position.

I am also taking care that it can be used in a deferred shading. Now I will create reflection and refraction inputs.

Will be sharing the news to the PR, suggestions, tests and enhancement are welcome 👍

[mrdoob]

Interesting!

[bhouston]

@sunag We are very interested in something like this. How can I help? I can add support for Standard at least.

[bhouston]

I am interested in creating fairly arbitrary graphs, so that the intermediate nodes also take inputs. So you can have a graph that looks like this:

A Texture(tex1, uv1)
B Texture(tex2, uv2)
C Blend(A,B, mode)
D Noise(param1, param2)
E Blend(C,D, mode)

And then use that final node E, as an input to a Material.

So it is very arbitrary, not limited to just textures.

[bhouston]

My goal would be to help create this in the next few weeks, hopefully collaborating with you in the next week or so if possible. I was looking at doing this via the shadergraph library here: #7339 But I am find creating it within ThreeJS directly. Really any solution is good as long as it is flexible and it works.

[bhouston]

I'm reading through your code, it is quite nicely designed. I have some initial feedback. Could I start a PR using your code as a base start collaborating on it?

(1) I'd have the material resolve the references. Basically references would have names that they would ask their material to resolve, and the material would give back a snippest of code on how to access that data. This would also allow the material to know what variables (uniform/varyings) are used the nodes, so it can optimize appropriately. This also allows for different materials to resolve references differently, thus making the nodes more portable, rather than having to know how to the materials implement things, especially when there are differences between fragment and vertex implementations.

(2) I try to use the GeometryContext object I created in the lights refactor, it gives consistent access to a lot of the required local variables. But of course that can be resolved by the material itself.

(3) I've have UV just be another reference, which is resolved by the material. And I would have NodeTexture should actually take a Node Input, thus allowing for procedurally generated UVs.

(4) I would call NodeCube, NodeTextureCube to be consistent with the rest of Three.JS. And I would remove the logic on how to actually go the ray casts from it. But I like the idea of standard cube maps, so I would actually not put the environment query for specular or diffuse in the nodes, but have that in the base phong material, and you can only control the normal used for the query, or the cubemap result itself (thus allowing it to be a procedurally determined color.) Does that make sense? So I would have the normal pluggable in the material and the cube texture pluggable (queriable by a direction and bias/lod, and returns a color). Thus one can provide a cube texture to the irradiance map and another to the specular map. We can swap out true sampleCube with @tschw's cubeToUV2 function as just a node swap.

(5) I'd try to add a NodeFunction that allows one to call arbitrary functions with parameters as an addition to your NodeOp (or maybe they could be merged in some fashion.)

(6) I'd get rid of all of the verbose NodeNormal, NodeTransform, NormalMap, etc individual class and just have some simple constructors that create a NodeReference with a name that is resolved by the material as appropriate. NodeReference could resolve uniforms, varyings as well as computed values in the shader.

(7) I do not understand the difference between NodeEnvironment and NodeCube. I think NodeEnvironment may be incomplete?

(8) It is confusing to have NodePhong not be derived from NodeMaterial. Although I see that NodeMaterial is derived from ShaderMaterial. I wonder if you called the direct derivative from ShaderMaterial, GraphMaterial (or NodeGraphMaterial) that would make more sense -- because all together the nodes form a graph, and it is the graph that becomes the material, not an individual node.

(9) I would suggest maybe some more varied terminology. I'd call the root node, MaterialNode, and one could derive PhongMaterialNode from it. I've have Vector3Node, FloatNode, etc derived from ValueNode -- not necessarily constant, but just a value. Thus one could pipe in three FloatNodes to a Vector3Node. I think you can have a helper that would make declaring each of these one line or so rather than the 10 or so currently.

[bhouston]

(10) I would move the name "Node" from the start of the class names to the back because that is how it is in the rest of the ThreeJS project.

(11) I would create the new MaterialNode class and it would be initialized with a list of uniforms and varyings. It would be default be able to resolve this and also track which it has resolved so one can track which features are needed. One could thus have a limited resolve in the derived PhongMaterialNode that would resolve the special cases and rely on the underlying class to do the simple ones (varyings, uniforms.)

(12) I am sort of confused between the difference between NodePhong and NodePhongMaterial. I didn't realize there was both until now.

[bhouston]

(13) there is code like this:

THREE.NodeGLPosition.prototype = Object.create( THREE.Node.prototype );
THREE.NodeGLPosition.prototype.constructor = THREE.NodeGLPosition;

THREE.NodeGL.prototype.generate = function( material, shader ) {

But above this snippet you already defined generate for NodeGL and you didn't define one for NodeGLPosition -- thus I think it is a copy-paste-edit error.

[bhouston]

(14) I would get rid of NodeReflectUVW and NodeRefractVector and instead just make this something one can request from the material via a Reference resolve. Calculating a reflection vector is straight forward. I have added it to GeometryContext in my unmerged experimental ThreeJS branches.

[bhouston]

(15) The way I would implement reflection and refraction would be to have them as color inputs on the Material. One would resolve Refect, ReflectLOD, and Refract, RefractLOD in the simple way you would resolve any variable, and then pass them into one's texture cube equivalent (procedural or samplerCube-based) and then pass the resulting color into Material. Is that how you were doing it?

[bhouston]

(16) I'm confused about the light input -- usually one doesn't have lights being pluggable, rather the light parameters are fully defined in the light class. I guess you need this additional flexibility? How do you envision it.

[sunag]

@bhouston woow, thank you very much feedback.
I will need several posts to answer :)

I am interested in creating fairly arbitrary graphs, so that the intermediate nodes also take inputs. So you can have a graph that looks like this:
A Texture(tex1, uv1)
B Texture(tex2, uv2)
C Blend(A,B, mode)
D Noise(param1, param2)
E Blend(C,D, mode)

Currently the syntax like this. Uv1 offset animate example:
I think that NodeMaterial to MaterialNode and THREE.PhongMaterialNode It would be better too.

var uv2 = false;
var uv_offset = new THREE.NodeFloat(0);     
var uv = new THREE.NodeOperator( '+', new THREE.NodeUV( uv2 ), uv_offset);
var texture = new THREE.NodeTexture( imgTexture, uv );

nodematerial.color = t;

// onUpdate
uv_offset.number += .01;

I think reverse the order with your suggestion get better (mode,A,B) to (A,B,mode). I am in the process of creating the reflex maps, cubemap and others...

The environment and Cubemap are incomplete.

Currently the bugs it may happen more because of the format converter still unfinished. This is responsible by vector conversion. vec3 to vec4 or vec4 for example.

https://github.com/sunag/sea3d/blob/gh-pages/Labs/Three.JS-NodeMaterial/index.html#L365

A Blend "texture" for example: ( I have not tested this code )
It can be implemented in the same of a THREE.NodeOperator

https://github.com/sunag/sea3d/blob/gh-pages/Labs/Three.JS-NodeMaterial/index.html#L1105

THREE.NodeBlend = function( a, b, mode ) {

    THREE.NodeInput.call( this, 'blend' );

    this.mode = mode;
    this.a = a;
    this.b = b;

};

THREE.NodeBlend.prototype = Object.create( THREE.NodeInput.prototype );
THREE.NodeBlend.prototype.constructor = THREE.NodeBlend;

THREE.NodeBlend.prototype.generate = function( material, shader, output ) {

    var a = this.a.build( material, shader, output );
    var b = this.b.build( material, shader, output );

    switch(this.mode)
    {
        case 'multiply':

            return this.format( '(' + a + '*' + b + ')', this.a.type, output);

            break;
    }

    return a;

};

.generate() is the responsible for the code generator. The calcs codes are stored in a cache if you want to use in more than one input without losing performance.

Still I do not set up pointers or constant for optimization...

The compilation is done by propagation in build() for vertex and fragment code.

I can put you as a collaborator? If you want to edit the code in any way, I will be working on it as well.

[bhouston]

I can put you as a collaborator? If you want to edit the code in any way, I will be working on it as well.

Thanks! I'll make PRs to yours so you can approve the changes.

I've added you (as well as @mrdoob, @WestLangley and @tschw) to a side project of mine that is attempting to define a set of reusable nodes and material definitions that can be transferrable between various renders. It is mappable onto this shader graph system you've created.

[bhouston]

I do not think you have to pay attention to the repo I just gave you access to if you do not want to. It is what I am interested in implementing on top of this.

[sunag]

(2) I try to use the GeometryContext object I created in the lights refactor, it gives consistent access to a lot of the required local variables. But of course that can be resolved by the material itself.

I wish that the lights are one LightNode. My concern is to harness the code already developed for Three.JS.

(3) I've have UV just be another reference, which is resolved by the material. And I would have NodeTexture should actually take a Node Input, thus allowing for procedurally generated UVs.

You can replace UV to a vec2 would it be this?

(5) I'd try to add a NodeFunction that allows one to call arbitrary functions with parameters as an addition to your NodeOp (or maybe they could be merged in some fashion.)

this would be great. mainly for a BlendNode.

(6) I'd get rid of all of the verbose NodeNormal, NodeTransform, NormalMap, etc individual class and just have some simple constructors that create a NodeReference with a name that is resolved by the material as appropriate. NodeReference could resolve uniforms, varyings as well as computed values in the shader.

In this line of thought I think the MaterialNode could be a base of material Phong and Physical material.

(7) I do not understand the difference between NodeEnvironment and NodeCube. I think NodeEnvironment may be incomplete?

I still can not finish these Nodes.

(8) It is confusing to have NodePhong not be derived from NodeMaterial. Although I see that NodeMaterial is derived from ShaderMaterial. I wonder if you called the direct derivative from ShaderMaterial, GraphMaterial (or NodeGraphMaterial) that would make more sense -- because all together the nodes form a graph, and it is the graph that becomes the material, not an individual node.

NodeMaterial would be the root node material, it is necessary to use a node for vertex and fragment. NodePhong is hibrid and NodePhongMaterial is only a proxy class. This can then be merged.

(9) I would suggest maybe some more varied terminology. I'd call the root node, MaterialNode, and one could derive PhongMaterialNode from it. I've have Vector3Node, FloatNode, etc derived from ValueNode -- not necessarily constant, but just a value. Thus one could pipe in three FloatNodes to a Vector3Node. I think you can have a helper that would make declaring each of these one line or so rather than the 10 or so currently.

Sounds good.

(16) I'm confused about the light input -- usually one doesn't have lights being pluggable, rather the light parameters are fully defined in the light class. I guess you need this additional flexibility? How do you envision it.

This would be for the lightmap or a possible LightNode.

[bhouston]

This would be for the lightmap or a possible LightNode.

I like the idea of a pluggable lightmap because one could define the UVs for it explicitly. :)

[sunag]

@bhouston Fix several corrections today in this file: But still has a lot to do.
https://github.com/sunag/sea3d/blob/gh-pages/Labs/Three.JS-NodeMaterial/three.node.js

This is the playground that I am creating 🎨 Textures and buttons is a drag and drop, works in chrome only

http://sea3d.poonya.com/flow/

[bhouston]

Amazing, stuff! Holy crap! It is beautiful.

Would it be possible to share the code in a way that I can also contribute? As a public PR or something?

[bhouston]

You are working with the Sea3D project here right?

https://github.com/sunag/sea3d/tree/gh-pages/Labs/Three.JS-NodeMaterial

So I can just fork it and start contributing? Would you accept PRs? How can we effectively collaborate.

I haven't asked but @mrdoob probably (?) would love to have this within the ThreeJS project itself.

[mrdoob]

Definitely!

[sunag]

So I can just fork it and start contributing? Would you accept PRs? How can we effectively collaborate.

Of course, I think your help would be amazing. I also have to bring other nodes types, like saturation, noise as you suggested.

You are working with the Sea3D project here right?

I think in making a PR for Three.JS with examples so all this is defined.

@mrdoob What do you about the materials names, THREE.MaterialNode or THREE.NodeMaterial?

[mrdoob]

It's a type of material, so it should be THREE.NodeMaterial.

[sunag]

Up. I think I'm near to a PR?
sunag/sea3d@55cf70a

Drag this cubemap to playground and any other texture for tests
https://raw.githubusercontent.com/mrdoob/three.js/master/examples/textures/skyboxsun25degtest.png

http://sea3d.poonya.com/flow/

[sunag]

a rim shader example

[sunag]

area reflection example

[bhouston]

This is so awesome @sunag!

[sunag]

@bhouston thanks! you think it will be difficult to convert to R74?

[GGAlanSmithee]

This is very impressive work! Reminds me of shaderforge.

Very good job so far!

[bhouston]

@sunag It will be a bit of work, but I would like to to help and most of the big structural changes in R74 shader code are my fault. :)

[bhouston]

@GGAlanSmithee it is also very similar to what UE4 has: https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/ExpressionReference/index.html

[pailhead]

@Usnul if you don’t like this you’ll free to refactor this? It doesn’t matter if it looks like this or not it’s just important for it to be built on top of this, is my understanding.
I don’t know though if refractors should happen while this is in examples (or possibly another repo) or should it be moved to src straightaway as @bhouston is suggesting.

[pailhead]

@IARI

I really don't know glsl very well, and also I find it - pardon my wording - a super pain in the arse to read and write shader code.

but judging on what I know, nodes would be the best way to do this in a truely elegant and extendable fashion.

I can't claim that i know GLSL very well either, but i know it enough to solve my problems. From what i know, i mostly agree with what @Usnul wrote here, and what @AndrewRayCode wrote before. I find the node thing to be verbose compared to GLSL.

If you don't know GLSL i think it would be very hard to compare it with something else. As you wrote it yourself, if we judge on what you know, and you said it yourself you don't know glsl i think your input should be taken with a disclaimer :)

It's like if i said:

Japan is the worst place on this planet... but i've never actually been to Japan".

Thank you.

[pailhead]

method names are obscure or downright misleading. (e.g. what do you think method "build" does? - make a guess)

Wow, I didn't even realize that this is a ShaderMaterial underneath. My guess is that this does all the stuff that THREE.WebGLRenderer does with shader templates.

[sunag]

I really don't know glsl very well, and also I find it - pardon my wording - a super pain in the arse to read and write shader code.
but judging on what I know, nodes would be the best way to do this in a truely elegant and extendable fashion.

@pailhead What he means is that there are intermediate levels between things.

I do not understand why this alarm. @mrdoob will never merger something other than a great solution. All great PR, improvements from anyone in NodeMaterial or not are very useful for community. If this still has not been merger in core it is because we need to improve with more work.

[donmccurdy]

@sunag is there anything particular others can do to help you with this project? I would be willing to write documentation, if you’re ready for docs and could review them.

[sunag]

@sunag is there anything particular others can do to help you with this project? I would be willing to write documentation, if you’re ready for docs and could review them.

This will be great. No doubt helps a lot.

[takahirox]

Node based material system looks amazing, it'd expand the power of expression.

How will we switch from existing material system to node based material system in core? Will we both two systems or replace the existing one with the node one?

Sorry, maybe I don't think I catch up this thread because this thread is too huge...

[sunag]

Will we both two systems or replace the existing one with the node one?

I see replacement as the ultimate goal, is better both for performance and maintenance. The next step is make a MeshStandardNodeMaterial and others node materials out in the core run exactly like the core in visual, syntax and performace.

[sunag]

WIP #14333

[claudioviola]

I've to show shadowMap in my project.

I've a scene with a directional light and 2 geometry that using MeshStandardNodeMaterial.

I've switched on
castShadow = true for light
castShadow and receivingShadow for both objects.
shadowEnabled = true for Renderer

Why doesn't works it?
Do I have to use shadow property of the MeshStandardNodeMaterial? Is this the purpose?

Anyone can help me?
Thanks!

[Usnul]

@claudioviola
support questions go to forum of stackoverflow.

[Mugen87]

@sunag Do you think NodeMaterial is stable enough so we can start with providing TypeScript declaration files? If you are not planning major API changes in the near future, I would start with this task since NodeMaterial is the last group of modules with no TS support.

[sunag]

@sunag Do you think NodeMaterial is stable enough so we can start with providing TypeScript declaration files? If you are not planning major API changes in the near future, I would start with this task since NodeMaterial is the last group of modules with no TS support.

@Mugen87 Yes, there must be more additions than changes. Add TS support will be great. 👍

[Mugen87]

Okay, TS support is ready with the next release R017 🙌

[Mugen87]

Closing. Better to track node material related issues in new threads.

[sdarpeng]

@sunag Why the nodes UV is not the same to geometry source UV?
the top face of the geometry lost its UV and it seems that the whole UV is inverted.

[sunag]

Hi @sdarpeng. This could be about Texture.flipY but I think better you create a new issue and share a live example in jsfiddle for these cases.