Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

glTF 2.0: texture packing for Metallic-Roughness? #857

Closed
donmccurdy opened this issue Feb 28, 2017 · 69 comments
Closed

glTF 2.0: texture packing for Metallic-Roughness? #857

donmccurdy opened this issue Feb 28, 2017 · 69 comments
Labels
2.0 PBR Physically Based Rendering resolved specification

Comments

@donmccurdy
Copy link
Contributor

From this paragraph, I see that the Metallic-Roughness workflow will make use of texture packing by assigning:

Channel Map
R Metallic
G Roughness
B
A

Were there plans for using the B channel later? If Occlusion is the obvious next choice, is it worth altering the order to match UE4's Occlusion/Roughness/Metallic R/G/B ordering?

I'm not very familiar with texture creation workflows, so maybe this is a non-issue and easy for exporters/converters to deal with. But thought I'd ask, as THREE.MeshStandardMaterial is already configured for that OcclusionRoughnessMetallic ordering. Thanks!

@pjcozzi
Copy link
Member

pjcozzi commented Feb 28, 2017

@mlimper?

@pjcozzi pjcozzi added 2.0 PBR Physically Based Rendering specification labels Feb 28, 2017
@mlimper
Copy link
Contributor

mlimper commented Feb 28, 2017

That actually sounds like a reasonable proposal - especially as it's already being used elsewhere like that.

The ordering should not be an issue IMHO, if we can make it fit existing conventions, that sounds like a plus.

Only thing we might want to make sure is that people can use the extension safely without having an occlusion map... which then probably means that somewhere inside the glTF material description we will want to specify whether the texture content has an occlusion channel or not.

@sbtron and colleagues, what do you think?

@lexaknyazev
Copy link
Member

I fully support merging them and defining static channel mapping at least for MR.
Texture swizzling isn't available in WebGL, so engines would need to maintain multiple versions of shader code to support different mappings.

How would SpecGloss model fit with such change?

@lexaknyazev
Copy link
Member

Without swizzling support, this would require shader regeneration to cover arbitrary cases.

@mlimper
Copy link
Contributor

mlimper commented Feb 28, 2017

One fixed mapping sounds OK. Why have multiple ones? Separate textures seems like an unnecessary step, given that we're aiming for a delivery format - it's not supposed to be authored as-is.

@sbtron
Copy link
Contributor

sbtron commented Mar 1, 2017

The idea with the additional maps in the core spec was to be able to share them across multiple material models so other material models defined through extensions could also use them. The spec gloss material defined in extension can also use the common occlusion map as its currently proposed - https://github.com/sbtron/glTF/tree/KHRpbrSpecGloss/extensions/Khronos/KHR_materials_pbrSpecularGlossiness#additional-maps.

Packing it with metal roughness while useful for metal roughness would make it unavailable to spec gloss or other future material extensions.

@emackey
Copy link
Member

emackey commented Mar 1, 2017

@sbtron But an exported spec/gloss model wouldn't want a loose occlusion map would it? It would be more useful to bundle the occlusion and gloss maps together for a spec/gloss model, I would think. The ability to deliver a tightly packed model is more in line with the goals of glTF than the ability to re-use loose textures across extensions, I would think.

@sbtron
Copy link
Contributor

sbtron commented Mar 2, 2017

@emackey the current specgloss texture doesn't have additional channels available as it is already packing spec (RGB) and gloss (A) together - https://github.com/sbtron/glTF/tree/KHRpbrSpecGloss/extensions/Khronos/KHR_materials_pbrSpecularGlossiness#specularglossinesstexture

@emackey
Copy link
Member

emackey commented Mar 2, 2017

Ah OK, understood. Even so, the spec/gloss extension could simply define its own occlusion map and not rely on the core one, right? A single model won't use both approaches in the same file, so each approach should be optimized for itself, as opposed to re-usability between approaches.

@bghgary
Copy link
Contributor

bghgary commented Mar 3, 2017

A single model won't use both approaches in the same file

The intention is that there can be both approaches in the same file. See https://github.com/sbtron/glTF/tree/KHRpbrSpecGloss/extensions/Khronos/KHR_materials_pbrSpecularGlossiness#conformance-and-best-practices

The example models have both at the same time. https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/BoomBox/glTF-pbrSpecularGlossiness

@emackey
Copy link
Member

emackey commented Mar 3, 2017

there can be both approaches in the same file.

My mistake again. Is there no way to implement this without causing pain to the spec/gloss workflow? Since the proposal is Occlusion (R), Roughness (G), Metallic (B), could we say that rough/metal (G/B) are optional when extensions are present, and a grayscale version of the image can be supplied offering only occlusion?

I do think there's some value in conforming to an existing RGB mapping instead of inventing a new one, albeit limited value, not worth messing up the spec/gloss extension.

That said, the original post above contains a link to Substance Painter export settings, and I just confirmed that Substance Painter 2.4 can have user-defined channel mappings on export. So even if we stick to a channel mapping that's unique to glTF, it's not unreasonable to expect existing authoring tools to produce it.

@lexaknyazev
Copy link
Member

Since the proposal is Occlusion (R), Roughness (G), Metallic (B), could we say that rough/metal (G/B) are optional when extensions are present, and a grayscale version of the image can be supplied offering only occlusion?

This also frees one texture unit. Possible layout with both models (note, that we still have unpopulated A channel in textures 2, 5, and 6):

  1. MR: BaseColor (RGB) + alpha-coverage (A)
  2. Occlusion (R) + Metallic-Roughness (GB)
  3. SpecGloss: Diffuse (RGB) + alpha-coverage (A)
  4. SpecGloss: Specular (RGB) + Glossiness (A)
  5. Normal - RGB (X/Y/Z)
  6. Emissive - RGB

One more possible caveat: we can't use JPEG for BaseColor / Diffuse textures, when they contain alpha channel. Is it OK with established workflows?

@bghgary
Copy link
Contributor

bghgary commented Mar 5, 2017

Another thing to consider is whether the occlusion channel is in sRGB or linear space. Unity imports models using a grayscale texture in sRGB space while Unreal uses the aforementioned packed structure but in linear space. Supposedly using linear can cause banding, but I still have yet to find any real evidence that supports this.

@javagl
Copy link
Contributor

javagl commented Mar 5, 2017

Although the compression for JPG heavily depends on the image contents, consider, for example, the baseColor texture of the BoomBox sample, and the following file types and sizes:

  • 3209 kB (original PNG, probably compression level 6)
  • 2347 kB (PNG with compression level 9)
  • 635 kB (JPG with quality 0.95)
  • 390 kB (JPG with quality 0.9)
  • 197 kB (JPG with quality 0.7)

When the spec dictates (for example, for Spec/Gloss case) that the images must have an alpha channel, and thus, that they must be stored as PNG, then the arguments for combining these channels into a single (namely, that "glTF is a transmission format and has to be compact") are rendered absurd by the fact that two images could be created which (essentially) contain the same information, but only have one tenth of the size of the proposed format.

Disclaimer: I don't have a clue about other image formats, KTX and whatnot. All I can say is that "common" textures as (highly-compressed) PNGs are still up to 10 times larger than a JPG with a high quality setting that is hardly distinguishable from the original PNG. So I'd at least advocate for not enforcing the use of PNG in the spec.

@emackey
Copy link
Member

emackey commented Mar 6, 2017

Lossy JPG, even at high quality settings, may not be the best format for metal/rough/occlusion. In JPG, high-contrast lines in a single color channel tend to bleed into the other two channels, because for normal images that's not too noticable to the eye. JPG lossiness is based on what human vision can't see in an image. When the 3 channels represent un-related linear values, such as occlusion, metal, and roughness, a high-contrast boundary in one channel may not be present in the other channels, and JPG could easly allow such details to bleed into the wrong channels. PNG is non-lossy compression and shouldn't have this particular problem (although browsers do a terrible job auto-converting color spaces in PNG files in uncontrollable ways, but I digress).

@lexaknyazev
Copy link
Member

In JPG, high-contrast lines in a single color channel tend to bleed into the other two channels,...

That's valid reason against JPGs for MR textures. What about baseColor + alpha-coverage?

although browsers do a terrible job auto-converting color spaces in PNG files in uncontrollable ways

An updated texture/image definitions PR explicitly demands disabling any browser conversions. Please see #860.

@emackey
Copy link
Member

emackey commented Mar 6, 2017

That's valid reason against JPGs for MR textures. What about baseColor + alpha-coverage?

JPG should be fine for basecolor RGB. It does not support Alpha, though.

Likewise, for a specular map (although I've demonstrated some spec/gloss ignorance in this thread already), I'd wager that an RGB specular map is essentially an image that JPG would preserve nicely. But again, the gloss map needs the alpha channel, which JPG does not offer. So that's a non-starter unless the gloss map is moved elsewhere.

I'm not sure what JPG would do to a normal map. At least it doesn't need the alpha channel, and the RGB channels are more directly related than a roughness channel is to a metallic channel. Even so I'm not sure what damage JPG's compression would do to vectors.

@javagl
Copy link
Contributor

javagl commented Mar 6, 2017

@emackey Sure, sharp edges may cause artifacts. And it's more likely that the "bleeding" will be more visible when it affects e.g. an emission channel than e.g. an occlusion channel. So in general, it's difficult to make a statement about how "noticable" they are for certain channels, certain textures and certain compression factors.

I'm just mentioning this, and leave it to others to either draw conclusions from that or not, but the point is: The potential size difference between PNG and JPG is huge. So huge that one Metallic/Roughness/Occlusion PNG may easily still be three times larger than three individual JPGs, each storing one channel...

@emackey
Copy link
Member

emackey commented Mar 6, 2017

The potential size difference between PNG and JPG is huge.

Agreed, but keep in mind, this size difference is only in terms of file size or network bandwidth, not GPU memory. Both formats result in un-compressed arrays of pixels in memory. So using three JPGs in place of a single PNG may save network traffic, but costs both texture memory and texture units.

Here's my impression of the various goals and priorities that have been discussed on this thread so far. Let me know if I got this all straight.

  • Need to have an option for metal/rough and spec/gloss to coexist in the same file, for demos and for non-HTTP deployments (file systems, network drives, etc.) where a single file image is provided to clients needing different workflows. Filesize may be not much of a concern here, since the author has chosen to double up on the workflows.

  • Need to have an option for either metal/rough or spec/gloss to exist on its own, without any baggage from the other format, for when the client workflow is known in advance or when a client has used HTTP/REST to specify which workflow they need. Reduced filesize is preferable here for low bandwidth deployments.

  • Need to have one official channel mapping per workflow, so that clients don't have to adapt to arbitrary user-defined mappings at runtime.

  • Would like to use minimal texture memory if possible. For mobile devices, this may be even more important than the network bandwidth concern.

  • Would like to use fewest number of texture units.

  • Would like to reduce network bandwidth for disadvantaged clients. This may imply avoiding the alpha channel where possible, so that the smaller JPG format can be used, at least where this doesn't conflict with other priorities listed above.

  • Might be nice to embrace existing standard channel mappings. This was the original point of this issue being opened, but now looks to me like the lowest of all of these priorities. At least some existing PBR authoring software is known to have user-definable export mappings, and glTF is expected to gain enough popularity that the tool authors will hopefully add the corresponding mappings to their tools.

@bghgary
Copy link
Contributor

bghgary commented Mar 7, 2017

I'm not an expert on loading images on the web. Is there an efficient way to load two images (say an RGB jpg and a grayscale jpg) into a single 4-channel texture (where the grayscale jpg is loaded into the alpha channel)?

I had always assumed that separating the images for each type of data would be an issue for texture bandwidth, but if it's efficient to load multiple images into one texture, then perhaps it's not an issue. If it is efficient, then maybe we should separate each type of data as different unpacked images so that jpg can be used when appropriate?

@lexaknyazev
Copy link
Member

All WebGL texture functions operate on one texture source at a time.
So one option is to manually manipulate ImageData before texture upload (JS loop through all pixels), other option is to use GPU and render-to-texture pre-pass to combine two textures.

Thoughts?

@bghgary
Copy link
Contributor

bghgary commented Mar 8, 2017

I propose that we move the discussion about png/jpg into a separate issue where I will try to quote / summarize what happened here. The original issue is regarding packing of the textures which we can continue to resolve here. Any objections?

@emackey
Copy link
Member

emackey commented Mar 8, 2017

I think the core issue here is selecting which channels go where, and unfortunately JPG has some influence on that by failing to provide a proper alpha channel, encouraging us to sway towards RGB textures instead of RGBA in the spec.

There's another reason (outside of JPG) to sway towards RGB as well: Many image manipulation software packages deliberately ignore or discard RGB data for pixels where alpha == 0. Sometimes this is done in the name of information security, preventing your paint program from "leaking" data that you though you erased from your image when you cleared the alpha channel for a certain range of pixels.

<anecdote> In fact, my very first encounter with a sample PBR model was one that had a roughness map wedged into the alpha channel of a texture with metallic and ambient occlusion (with the blue channel unused!) and this caused me a fair amount of grief attempting to extract the original channels without them getting premultiplied by the roughness channel. </anecdote>

Obviously this is not a problem for the baseColor or diffuse map (as the alpha, if supplied, is intended to be used the traditional way where zero values mean discard). I suspect this may not be a problem for Spec/Gloss (Does the spec color matter at all when gloss == 0? I would expect it to not matter). But I would advise against dumping occlusion into some random alpha channel of an unrelated map.

option is to use GPU and render-to-texture pre-pass to combine two textures

Now this is an interesting option: Any given glTF runtime implementation has the option to do this kind of pre-pass on any glTF assets, to make them conform to the local engine's needs. If an engine wants the occlusion channel jammed into the alpha channel of the normal map, sure, do it in a prepass step, the glTF spec doesn't mind at all. glTF is a "transmission format" spec after all, we should worry less about what layout the engine needs internally and more about how to best pack these assets for transmission, with the knowledge that engines can unpack and rearrange assets at will. I could imagine a prepass step to convert any metal/rough-only assets to spec/gloss, and then offering support of both types of glTF while internally only needing a spec/gloss PBR shader with a condensed set of maps.

So in summary

After all this discussion, I'm still leaning back to the same idea we had a few days ago for the transmission format, now with the knowledge that individual engines have the option to remap all this during a pre-pass step if they don't like the delivered channel mappings.

  1. MR: BaseColor (RGB) + alpha-coverage (A)
  2. Occlusion (R) + Metallic-Roughness (GB)
  3. SpecGloss: Diffuse (RGB) + alpha-coverage (A)
  4. SpecGloss: Specular (RGB) + Glossiness (A)
  5. Normal - RGB (X/Y/Z)
  6. Emissive - RGB

All of these alpha channels are optional (for models that don't use translucency, which no web developer expects JPG to provide), with the exception of (4) the glossiness one. But that's in an extension, so if someone thought it was critical to avoid, they could avoid the extension, and produce a core glTF 2.0 file with no alpha channels at all. Their engine could remap the channels at load time and maybe save a texture unit or two without their knowledge. Thoughts?

@emackey
Copy link
Member

emackey commented Mar 8, 2017

Whoops, correction on (2) above:

Occlusion is (R), Roughness is (G), and Metallic is (B), per the original post in this issue.

@donmccurdy
Copy link
Contributor Author

I don't know what the effect is, but that concern makes sense.

Would it be kicking the can down the road to change the metallicRoughness texture to use the b and g channels respectively, with no other immediate changes to the spec?

Original post notwithstanding, I don't have a strong preference here. Wanted to suggest UE4's ordering as nice-to-have in the absence of other considerations, of which there are apparently plenty. 🙂

@emackey
Copy link
Member

emackey commented Mar 10, 2017

the asset only has metallic-roughness, then it behaves one way, but if the asset also has specular-glossiness, then it behaves in a different way

Hmm, I don't think so. With @donmccurdy's most recent proposal, the difference is really whether or not occlusion shares an index with metalRough, regardless of the presence of other workflows.

I like the strategy of saying occlusion is always r and rough/metal, if present, is always g/b, and the author then has the option of providing separate maps or using the same index to reference the same map when that's possible.

Does that resolve any of the concerns here?

@emackey
Copy link
Member

emackey commented Mar 10, 2017

Wanted to suggest UE4's ordering as nice-to-have

The combination of a grayscale with a 2-channel texture is also a nice-to-have in my humble opinion 😄

@bghgary
Copy link
Contributor

bghgary commented Mar 10, 2017

@donmccurdy's most recent proposal

If you look carefully at the proposal, there is a difference between "metallic-roughness only" and "both workflows" in that occlusion is stored in a separate texture for the latter. Sampling one texture vs sampling two textures in the shader will have different performance characteristics.

@emackey
Copy link
Member

emackey commented Mar 10, 2017

occlusion is stored in a separate texture for the latter

Sure, but that's only because the index is different there, no? What's to stop a user from specifying different index values for a metal/rough-only glTF, or specifying the same index twice in a dual-workflow glTF?

Does the current spec forbid reuse of index values? I could imagine a test model where the normal map index was copy-pasted to the base color index, resulting in an oddly-purple-colored model, but is there anything technically preventing re-use of index values like this?

I'm saying I don't think this is a special-case of index value re-use. It's just a case where it makes sense to do so.

@donmccurdy
Copy link
Contributor Author

donmccurdy commented Mar 10, 2017

Sampling one texture vs sampling two textures in the shader will have different performance characteristics.

Hm, let's assume 2 textures with r and gb channels give strictly worse performance characteristics than 1 texture with rgb channels. This seems realistic, considering browsers impose maximum concurrent connections per host, which I believe was ~6 in 2015.

If we stick with the current spec, we will have imposed this lower-performance scenario on all Metal-Rough assets. Alternatives to this appear to be:

  1. Combine into occlusionRoughnessMetallicTexture, giving consistent and better performance to Metal-Rough, but worse performance to Spec-Gloss. Spec-Gloss must read 1 channel of a 3-channel texture when both workflows are provided.
  2. Separate roughnessMetallicTexture gb and occlusionTexture r, which may or may not share an index. If both workflows are provided, the lower-performance Metal-Rough scenario applies.
  3. Allow the KHR_materials_pbrSpecularGlossiness extension to specify occlusionTexture (whenever available, or as an override to the material's default). Authoring tools exporting both workflows may now choose between the downside of (1), the downside of (2), or creating a second copy of the occlusion data for optimal performance but less convenience. Example:
"materials": [
    {
        "pbrMetallicRoughness": {
            "roughnessMetallicTexture": {
                "index": 1
            }
        },
        "occlusionTexture": {
            "strength": 0.75,
            "index": 1
        },
        "extensions": {
            "KHR_materials_pbrSpecularGlossiness": {
                "specularGlossinessTexture": {
                    "index": 2
                },
                "occlusionTexture": {
                  "strength": 0.75,
                  "index": 3
                }
            }
        }
    }
]

I can't think of a 4th combination, unless we bundle occlusion with a less-common data type.

@bghgary
Copy link
Contributor

bghgary commented Mar 10, 2017

Thanks @donmccurdy. I can't think of another combination either.

We have 4 choices where all of them have downsides. None of the alternatives jump out to me as the solution that just works. Personally, I prefer consistent behavior over more complicated solutions, considering performance trade-offs. And given that we already have multiple implementations out there, I would want a solid solution to switch to.

Thoughts?

@emackey
Copy link
Member

emackey commented Mar 10, 2017

I think the metal/rough-only workflow could benefit from a slight change to the 2.0 spec as it stands today:

  • Move the metallic channel out of r and over to b.
  • Rename accordingly (metallicRoughness to roughnessMetallic).
  • Specify that occlusion explicitly comes only from the r channel of its texture, in case a color texture is supplied.
  • No change to spec/gloss extension.

@donmccurdy
Copy link
Contributor Author

My own preference would be to do the rb channel change as @emackey suggests, as glTF 2.0 is not yet finalized and the cost of such a change will only go up. That specific option does not seem more complex to me; its cost is the requirement of updating multiple existing implementations.

If that updating cost is already too high, then one more option along the same lines:

  • No change to metallicRoughness.
  • Specify that occlusion explicitly comes only from the b channel of its texture, in case a color texture is supplied.
  • No change to spec/gloss extension.

This is backwards-compatible with existing authoring tools, but at least leaves open the possibility of packing metallic + roughness + occlusion later.

@bghgary
Copy link
Contributor

bghgary commented Mar 18, 2017

Regarding the JPG/PNG discussion, there is a tool called pngquant that quantized PNGs into a palette 8-bit image which reduces the size a lot. Note that I had to modify the code a bit to make it not automatically premultiply alpha. Quality settings for JPG and quantized PNG are both 90.

Results:

  • Spec-gloss quantizes to png with almost no loss
  • Base color / diffuse with alpha coverage quantizes to png with minimal loss.

See the difference for yourself.

BoomBox - https://sbtron.github.io/BabylonJS-glTFLoader/?imageFormat=jpg-with-quantized-png&model=BoomBox

FarmLandDiorama - https://sbtron.github.io/BabylonJS-glTFLoader/?imageFormat=jpg-with-quantized-png&model=FarmLandDiorama

I've also included a per-channel pixel diff between PNG and PNG-quantized in the corresponding quantized folders of each model.

BoomBox - https://github.com/sbtron/BabylonJS-glTFLoader/tree/master/models/2.0/jpg-with-quantized-png/BoomBox/glTF-pbrSpecularGlossiness/diff

FarmLandDiorama - https://github.com/sbtron/BabylonJS-glTFLoader/tree/master/models/2.0/jpg-with-quantized-png/FarmLandDiorama/glTF-pbrSpecularGlossiness/diff

Here are some stats for BoomBox and FarmLandDiorama models:

File alpha png jpg / png jpg / png_quantized
BoomBox_baseColor no 3,285,844 432,168 432,168
BoomBox_diffuse no 3,210,781 419,396 419,396
BoomBox_emissive no 132,833 72,502 72,502
BoomBox_metallicRoughness no 3,141,233 556,917 556,917
BoomBox_normal no 2,845,923 203,361 203,361
BoomBox_occlusion no 1,659,622 156,478 156,478
BoomBox_specularGlossiness yes 3,648,876 3,648,876 751,990
FarmLandDiorama_baseColor no 1,988,194 394,237 394,237
FarmLandDiorama_baseColor1 no 1,426,211 218,012 218,012
FarmLandDiorama_baseColor2 no 1,670,131 262,972 262,972
FarmLandDiorama_baseColor3 yes 2,263,162 2,263,162 634,350
FarmLandDiorama_baseColor4 yes 1,391,035 1,391,035 449,720
FarmLandDiorama_diffuse no 1,988,194 394,237 394,237
FarmLandDiorama_diffuse1 no 1,418,602 217,773 217,773
FarmLandDiorama_diffuse2 no 1,679,801 281,477 281,477
FarmLandDiorama_diffuse3 yes 2,263,162 2,263,162 634,350
FarmLandDiorama_diffuse4 yes 1,391,035 1,391,035 449,720
FarmLandDiorama_metallicRoughness no 997,623 190,199 190,199
FarmLandDiorama_metallicRoughness1 no 698,596 109,776 109,776
FarmLandDiorama_metallicRoughness2 no 770,394 153,362 153,362
FarmLandDiorama_metallicRoughness3 no 15,777 17,014 17,014
FarmLandDiorama_metallicRoughness4 no 375,960 53,053 53,053
FarmLandDiorama_normal no 4,321,064 690,771 690,771
FarmLandDiorama_normal1 no 875,489 235,055 235,055
FarmLandDiorama_normal2 no 611,220 117,596 117,596
FarmLandDiorama_normal3 no 1,419,327 266,866 266,866
FarmLandDiorama_normal4 no 1,158,902 277,548 277,548
FarmLandDiorama_occlusion no 54,431 16,815 16,815
FarmLandDiorama_occlusion1 no 42,991 10,203 10,203
FarmLandDiorama_occlusion2 no 28,193 8,453 8,453
FarmLandDiorama_occlusion4 no 45,015 10,223 10,223
FarmLandDiorama_specularGlossiness yes 1,053,689 1,053,689 344,075
FarmLandDiorama_specularGlossiness1 yes 763,818 763,818 111,771
FarmLandDiorama_specularGlossiness2 yes 890,005 890,005 164,561
FarmLandDiorama_specularGlossiness3 yes 20,541 20,541 235
FarmLandDiorama_specularGlossiness4 yes 418,164 418,164 27,769

@lexaknyazev
Copy link
Member

Slightly related - perceptual JPEG encoder:
https://github.com/google/guetzli/

@bghgary
Copy link
Contributor

bghgary commented Mar 30, 2017

Here is what I'm testing out with the packing:

For assets without the KHR_materials_pbrSpecularGlossiness extension:

"materials": [
    {
        "pbrMetallicRoughness": {
            "metallicRoughnessTexture": {
                "index": 1
            }
        },
        "occlusionTexture": {
            "strength": 0.75,
            "index": 1
        }
    }
]

For assets with the KHR_materials_pbrSpecularGlossiness extension:

"materials": [
    {
        "pbrMetallicRoughness": {
            "metallicRoughnessTexture": {
                "index": 1
            }
        },
        "occlusionTexture": {
            "strength": 0.75,
            "index": 2
        },
        "extensions": {
            "KHR_materials_pbrSpecularGlossiness": {
                "specularGlossinessTexture": {
                    "index": 3
                }
            }
        }
    }
]

Here are the implementation changes required to do this in BabylonJS:
bghgary/Babylon.js@0ed5ae0
bghgary/Babylon.js@010e955

Here are the changes to the models:
sbtron/BabylonJS-glTFLoader@21fc731

@emackey
Copy link
Member

emackey commented Mar 30, 2017

@bghgary That looks good to me with one possible typo: Did you mean to use metallicRoughnessTexture in the first sample, and roughnessMetallicTexture in the second?

Looks like the sample models stick with metallicRoughnessTexture in your last link there, that seems fine.

@bghgary
Copy link
Contributor

bghgary commented Mar 30, 2017

Beat me to the punch. Was about to reply with this.

Implementing this change in the exporter was slightly more complicated, but it wasn't terrible.
Here is the proposed changes (slightly changed from the proposals above):

  • Move metallic from r to b
  • Leave metallicRoughnessTexture the same name
  • Occlusion is sampled only from the r channel
  • Occlusion is stored in linear space
  • Spec-gloss extension or other extensions that want to share the occlusion texture should store the occlusion texture separately

+1 if you agree and we can close on this issue.

@bghgary
Copy link
Contributor

bghgary commented Mar 30, 2017

Did you mean to use metallicRoughnessTexture in the first sample, and roughnessMetallicTexture in the second?

No, fixed the second one.

@donmccurdy
Copy link
Contributor Author

This looks great, thanks @bghgary! We've got an implementation and demo for metal-rough in three.js now. Still tracking down a visual artifact. Spec-gloss in progress, thanks to @takahirox.

@bhouston
Copy link
Contributor

bhouston commented Apr 3, 2017

Thanks @donmccurdy for pushing glTF to be compatible with my change to ThreeJS here from January: mrdoob/three.js#10691 This is awesome stuff and nice to see everything converging on best practice. Hopefully this standard can spread to Unity in the future.

@AurL
Copy link

AurL commented Apr 4, 2017

Thanks @donmccurdy @bghgary, we will apply the update on our side.

@AurL
Copy link

AurL commented Jun 2, 2017

@bghgary I'm back here to ask about something that is unclear for me (and sorry in advance if I missed something)

How should we know if we should use the R channel or not ? Or should we always use it and then have a full white channel by default ?

Looking at the samples Avocado vs BoomBox, I understand that we should take the R channel (occlusion) into account only if occlusionTexture if specified, but it seems not to be compliant with your comment here ?
It's ok if the sample is compliant, but if it's not, how should we know if the R channel should be taken into account or not ?
(if you look at the Avocado sample seems to have a full black R channel, so using it as occlusion is wrong)

@AurL
Copy link

AurL commented Jun 2, 2017

Well, sorry for the dumb question, I have the answer.

@pjcozzi
Copy link
Member

pjcozzi commented Jun 15, 2017

Updated in #826

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.0 PBR Physically Based Rendering resolved specification
Projects
None yet
Development

No branches or pull requests