Improving map efficient / compatibility with Substance Designer

[bhouston]

Description of the problem

We have been researching using Substance for creating Three.JS materials. One interesting thing I found out is that UE4 uses packed textures to represent AO, Roughness and Metalness in a single texture -- basically R for Ambient Occlusion, G channel for Roughness and B for Metalness (source: https://support.allegorithmic.com/documentation/display/SPDOC/Unreal+Engine+4 ). This is a great memory saving as compared to having three separate PNGs loaded.

I have also noticed that Unity expects roughness to be encoded as the Alpha parameter of a texture map, and often Substance Designer will export RGB as 0,0,0 in that case. This is not compatible with Three.JS in its current form.

One way to deal with all of the above issues is to add to Three.JS's materials the ability to specify which channel of a RGBA map should be used for which parameter. Thus one could specify that the "R" channel of a map should be accessed for AO, the "G" channel be used for Roughness and the "B" channel for Metalness. This would also enable exact compatibility with both UE4 and Unity material formats.

Implementation would allow for this flexilibility on a per-material basis - because some maps may be UE4 style while others are Unity style, or classic Three.JS style (always read "R" channel.)

In the shaders I would suggest these channels be specified via defines because they do not need to change on frequent basis.

On the material, I guess would add something like a roughnessMapChannel to go along with roughnessMap? aoMapChannel to go along with aoMap? And metalnessMapChannel to go along with metalnessMap?

Channel will be I guess some constants like THREE.RChannel, THREE.BChannel, etc.? Or THREE.R?

/ping @WestLangley

[bhouston]

I guess one could also make these global settings that affect all materials. I am okay with either approach. But having it configurable is probably a good idea, especially because it allows for one texture to be used for AO, Roughness and Metalness at once.

[bhouston]

Or one quick fix we could do is just have aoMap access "r", roughness access "g", and metalness access "b". This would match UE4 and it would be completely backwards with users who are currently using grayscale maps. This would increase our potential texture efficiency by 3x in this regards and it would introduce Substance compatibility via being compatible with UE4's packed texture format.

[WestLangley]

@bhouston Great idea. I think the reason we require separate maps is because there was no standard way of packing them.

I guess one could also make these global settings that affect all materials.

Global settings are slightly restrictive, but I think they would be OK. Something like:

THREE.Channels.roughnessMap = THREE.AChannel;

THREE.Channels.alphaMap = THREE.GChannel; // the current default, btw

How we pack that data and push it to the shader is an implementation detail I'll defer to someone else.

[mrdoob]

Hmm, I guess it's time to implement something like THREE.Map:

THREE.Map = function ( texture ) {
    this.uuid = _Math.generateUUID();
    this.texture = texture;
    this.channels = THREE.RGBChannels;
};

Or add channels to THREE.Texture and implement THREE.Image. So image will no longer point to a <img> or <canvas> but be wrapped by a THREE.Image with its own uuid.

var image = new THREE.Image( <img> );

var texture1 = new THREE.Texture( image );
texture1.channels = THREE.RChannels;

var texture2 = new THREE.Texture( image );
texture2.channels = THREE.BChannels;

var material = new THREE.MeshStandardMaterial( {
    map: texture1,
    aoMap: texture2
} );

I think the second option requires less changes in the serialisation.

[bhouston]

I think my last comment above were we just standardize on UE4 approach with aoMap as R, roughness as G and metallic as B is an easy change to make and it may be better to have a Three.JS standard that other tools like Substance could adopt. In some ways I worry that if we make it very flexible it will make it harder to create content for Three.JS.

[mrdoob]

Ah, interesting... I guess didn't understood your previous comment.

It's backwards compatible. So, hardcoding sounds good to me! 👍

[RemusMar]

Or one quick fix we could do is just have aoMap access "r", roughness access "g", and metalness access "b".
This would match UE4 and it would be completely backwards with users who are currently using grayscale maps.

+1

[sjlynch]

Did this ever get resolved? I'd like to try the free trial of substance painter and then play around with the materials in threejs but I'd like to hold off until I know it's supported.

[bhouston]

yes it was.

[bhouston]

In fact I believe glTF 2.0 went so far as to change their encoding to match as well.