Skip to content

Commit

Permalink
fixed exposure bug and added figures (#4648)
Browse files Browse the repository at this point in the history 
* fixed exposure bug and added figures

* link docs in FAQ
  • Loading branch information
@elalish
elalish authored Jan 17, 2024
1 parent 2da8180 commit 81b2f92
Show file tree
Hide file tree
Showing 7 changed files with 135 additions and 59 deletions.
3 changes: 2 additions & 1 deletion packages/model-viewer/src/three-components/Renderer.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -435,8 +435,9 @@ export class Renderer extends
const exposureIsNumber =
typeof exposure === 'number' && !Number.isNaN(exposure);
const env = element.environmentImage;
const sky = element.skyboxImage;
const compensateExposure = toneMapping === CustomToneMapping &&
(env == null || env === 'neutral' || env === 'legacy');
(env === 'neutral' || env === 'legacy' || (env == null && sky == null));
this.threeRenderer.toneMappingExposure =
(exposureIsNumber ? exposure : 1.0) *
(compensateExposure ? COMMERCE_EXPOSURE : 1.0);
Expand Down
Binary file added packages/modelviewer.dev/assets/ACESset.glb
View file
Binary file not shown.
Binary file added packages/modelviewer.dev/assets/CommerceSet.glb
View file
Binary file not shown.
2 changes: 1 addition & 1 deletion packages/modelviewer.dev/data/docs.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -887,7 +887,7 @@
"htmlName": "toneMapping",
"description": "Selects the function that compresses the HDR rendering to an SDR image on your screen. ACES is a film industry standard that is commonly used, though it has serious color-accuracy problems. AgX is a new and improved tone mapper seeing broad adoption in film and games.Commerce is a function designed specifically for accurate color reproduction in e-commerce. Our current default is and has been ACES, but in v4.0 this default will change to Commerce.",
"links": [
"<a href=\"../examples/lightingandenv/#renderExposure\"><span class='attribute'>exposure</span> example</a>"
"<a href=\"../examples/lightingandenv/#toneMapping\"><span class='attribute'>tone-mapping</span> example</a>"
],
"default": {
"default": "aces",
Expand Down
10 changes: 10 additions & 0 deletions packages/modelviewer.dev/data/faq.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,16 @@
"<a href='../fidelity'>glTF fidelity comparison</a>"
]
},
{
"name": "How do I ensure accurate colors?",
"htmlName": "color",
"description": "The short answer is to use neutral (grayscale) lighting and our new 'commerce' tone-mapping function, which is designed specifically to ensure faithful rendering of the glTF's colors. This is a complicated topic, so follow the links for much more in-depth explanations. The single biggest problem with color in 3D e-commerce is the industry-standard ACES tone mapper, so please consider using our commerce alternative.",
"links": [
"<a href='../examples/lightingandenv/#toneMapping'><span class='attribute'>tone-mapping</span> example</a>",
"<a href='../examples/color'>Achieving Color-Accurate Presentation with glTF</a>",
"<a href='../examples/tone-mapping'>Tone Mapping Considerations for Physically-Based Rendering</a>"
]
},
{
"name": "Why am I getting CORS errors?",
"htmlName": "cors",
Expand Down
38 changes: 29 additions & 9 deletions packages/modelviewer.dev/examples/color.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="shortcut icon" type="image/png" href="../assets/favicon.png"/>
<link type="text/css" href="../styles/examples.css" rel="stylesheet" />
<script type='module' src='https://modelviewer.dev/node_modules/@google/model-viewer/dist/model-viewer.min.js'></script>
<script type='module' src='../../../node_modules/@google/model-viewer/dist/model-viewer.js'></script>
<script defer src="https://web3dsurvey.com/collector.js"></script>
<script>
window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date;
Expand Down Expand Up @@ -404,17 +404,17 @@ <h3 id="photography">How does rendering compare to photography?</h3>
skews (blue to purple, red to orange) and saturation loss. AgX is a newer
and better tone mapper that holds hue better, but still has significant
contrast and saturation loss, which is desirable for its intended use cases
in games and film. For detail, please see our <a
href="tone-mapping.html">technical document</a> on the tradeoffs in tone
mapping and how our Commerce tone mapper was designed.</p>
in games and film. For detail, please see our technical document on the
tradeoffs in <a href="tone-mapping.html">tone mapping</a> and how our
Commerce tone mapper was designed.</p>

<figure>
<model-viewer
id="tone-mapping"
src="../assets/ShopifyModels/Mixer.glb"
tone-mapping="commerce"
camera-controls
alt="Tone mapping comparisons for difference 3D models"
alt="Tone mapping comparisons for different 3D models"
>
<p>Tone Mapping:
<select id="tone">
Expand All @@ -432,7 +432,7 @@ <h3 id="photography">How does rendering compare to photography?</h3>
</select>
</p>
</model-viewer>
<figcaption>Comparison of tone mapping functions for difference models.</figcaption>
<figcaption>Comparison of tone mapping functions for different models.</figcaption>
</figure>

<h3 id="validate-render">How do we validate a glTF 3D render?</h3>
Expand Down Expand Up @@ -471,15 +471,15 @@ <h3 id="validate-render">How do we validate a glTF 3D render?</h3>
post-processing step. Some things don't apply to rendering, like masking out
shadows to make them semi-transparent - a 3D renderer can do this
automatically. In addition to color-neutral tone mapping, sometimes the
colors are intentionally "corrected". Why? After all, if the measured light
colors are intentionally "corrected". Why? After all, if the captured light
from the actual scene is not correct, what is?</p>

<p>It may be that some of the post-processing color correction is simply a
matter of expediency. Before digital photography, to get the right look, the
lighting and scene had to be adjusted, which involves manual labor. Digital
post-processing can allow the lighting to be less precise. However in 3D
rendering, the lighting is equally digital, so it is generally a better
practice to keep the post-processing step simple (e.g. just ACES tone
practice to keep the post-processing step simple (e.g. just tone
mapping) and adjust the environment image if necessary. Since 3D rendering is
happening completely automatically in real time, there is no way to make
manual bespoke color adjustments for each frame.</p>
Expand Down Expand Up @@ -577,7 +577,7 @@ <h3 id="takeaway">What's the takeaway?</h3>
material baseColor, which is what allows the demo below to be compellingly
realistic.</p>

<figure>
<figure>
<model-viewer
id="environments"
src="../assets/ShopifyModels/Mixer.glb"
Expand All @@ -592,6 +592,26 @@ <h3 id="takeaway">What's the takeaway?</h3>
realistic.</figcaption>
</figure>

<p>Still, for most e-commerce vendors that are likely managing separate
contracts for 3D model generation and interactive website design, simplicity
is key to cost-effectiveness. My recommendation is to use our Commerce tone
mapping and if you already have approved marketing colors for your products,
then tell your artists to use those for the baseColor. Use a grayscale
lighting environment, adjusting if necessary to place highlights and change
exposure.</p>

<p>If the color is not known, but being matched by eye, then ensure the
artist's tools also use the Commerce tone mapper and a lighting environment
that is as close as possible to the production grayscale lighting
environment. Since our Commerce tone mapper is relatively new, it may not be
available in tools; in this case the next best thing is to turn off tone
mapping entirely. Make sure to educate your artists on the differences they
can expect - blown out highlights with hue skews, and saturation loss for
dark colors. Ensure they also frequently test their output in a
near-production environment, hopefully using &lt;model-viewer&gt; with
<code>tone-mapping="commerce"</code>. This way they can calibrate and ensure
the end user will see realistic colors.</p>

</div>
<div style="margin-top:24px"></div>
<div class="footer">
Expand Down
141 changes: 93 additions & 48 deletions packages/modelviewer.dev/examples/tone-mapping.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="shortcut icon" type="image/png" href="../assets/favicon.png"/>
<link type="text/css" href="../styles/examples.css" rel="stylesheet" />
<script type='module' src='https://modelviewer.dev/node_modules/@google/model-viewer/dist/model-viewer.min.js'></script>
<script type='module' src='../../../node_modules/@google/model-viewer/dist/model-viewer.js'></script>
<script defer src="https://web3dsurvey.com/collector.js"></script>
<script>
window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date;
Expand Down Expand Up @@ -146,19 +146,19 @@ <h3 id="purpose">The purpose of tone mapping</h3>
and even things like sepia can all be combined into a single resulting
function that here we're calling tone mapping. However, we are only
interested in hue-neutral functions and those are the only type of
tone mapping functions we'll be discussing here. Therefore the focus will be
primarily on luma, or brightness.</p>
tone mapping functions we'll be discussing here.</p>

<p>The tone mapping function used by <code>&lt;model-viewer&gt;</code> is
ACES, which is a standard developed by the film industry and is widely used
for 3D rendering. Like most tone mapping curves, it is fairly linear in the
central focus of its contrast range, then asymptotes out to smoothly
<p>The default tone mapping function used by
<code>&lt;model-viewer&gt;</code> has been ACES, which is a standard
developed by the film industry and is widely used for 3D rendering, though
it has some problems. Like most tone mapping curves, it is fairly linear in
the central focus of its contrast range, then asymptotes out to smoothly
compress the long tails of brights and darks into the required zero to one
output range, the idea being that humans perceive less difference between
over-bright and over-dark zones as compared to the bulk of the scene.
However, since some output range is reserved for these extra-bright
highlights, the range left over to represent the input range of matte
baseColors is also reduced somewhat. This is why the paper-white sphere does
baseColors is also reduced. This is why a paper-white material does
not produce white pixels.</p>

<p>Sometimes when working with matte objects and trying to compare output
Expand Down Expand Up @@ -186,28 +186,30 @@ <h3 id="purpose">The purpose of tone mapping</h3>
<figcaption>Toggle ACES tone mapping to see the difference it makes.</figcaption>
</figure>

<p>Note that once again the shiny and matte white plastic spheres are
indistinguishable, even without cranking up the exposure. Since half of the
matte white sphere is now rendering pure white, there is no headroom for
shiny highlights. Likewise, the top half of the sphere loses its 3D
appearance since the shading was removed by clamping the values. Tick the
checkbox to go back to ACES tone mapping for a quick comparison. Remember to
look away and back again after switching; another trick of human perception
is how dependent it is on anchoring. The yellow will look washed-out
immediately after switching from saturated yellow, but this perception fades
after looking around.</p>
<p>This model has six spheres with uniform materials: The top row are white
(baseColor RGB: [1, 1, 1]), while the bottom row are yellow (baseColor RGB:
[1, 1, 0]). From left to right they are shiny metal (metalness: 1,
roughness: 0), shiny plastic (metalness: 0, roughness: 0), and matte plastic
(metalness: 0, roughness: 1). The left-most can be thought of approximately
as polished silver and gold.</p>

<p>Tick the checkbox to remove tone mapping for a quick comparison. Note
that the shiny and matte white plastic spheres are now indistinguishable.
Since half of the matte white sphere is now rendering pure white, there is
no headroom for shiny highlights. Likewise, the top half of the sphere loses
its 3D appearance since the shading was removed by clamping the values.
</p>

<p>This example also highlights a second key element of good tone mapping
functions: desaturating overexposed colors. Look at the golden sphere
(lower-left) and compare to the previous version with ACES tone mapping
applied. The baseColor of a metal multiplies the incoming light, so a white
light on a golden sphere produces a yellow reflection (fully saturated
yellow, in this case of a fully saturated baseColor). With clamped tone
mapping, the highlight is indeed saturated yellow, but this does not look
perceptually right, even though you could make the argument it is physically
correct.</p>

<p>Good tone mapping curves like ACES not only compress the luma, but also
(lower-left) and compare to when ACES tone mapping is applied. The baseColor
of a metal multiplies the incoming light, so a white light on a golden
sphere produces a yellow reflection (fully saturated yellow, in this case of
a fully saturated baseColor). With clamped tone mapping, the highlight is
indeed saturated yellow, but this does not look perceptually right, even
though you could make the argument it is physically correct.</p>

<p>Tone mapping curves like ACES not only compress the luma, but also
push colors toward white the brighter they are. This is why the highlights
on the golden sphere become white instead of yellow. This follows both the
behavior of camera sensors and our eyes when responding to overexposed
Expand Down Expand Up @@ -270,13 +272,20 @@ <h3 id="#tradeoffs">Tradeoffs</h3>

<figure>
<model-viewer
src="../../shared-assets/models/silver-gold.gltf"
tone-mapping="aces"
id="reachable"
src="../assets/ACESset.glb"
camera-orbit="150deg auto auto"
camera-controls
alt="3D model of six example material spheres"
alt="3D model of ACES/Commerce tone mapping reachable colors."
>
<p>
<select id="set">
<option value="../assets/ACESset.glb">ACES</option>
<option value="../assets/CommerceSet.glb">Commerce</option>
</select>Tone Mapping Function
</p>
</model-viewer>
<figcaption>The ACES reachable colors.</figcaption>
<figcaption>Comparison of the ACES and Commerce tone mapping reachable colors. The cube represents the [0, 1] space in linear light - no sRGB curve has been applied.</figcaption>
</figure>

<p>Note that canary yellow, bright greens and blues are all impossible to
Expand Down Expand Up @@ -398,7 +407,10 @@ <h3 id="#commerce">Commerce tone mapper</h3>
only other parameter in this tone mapper controls the rate of desaturation,
which I chose as 0.15, which is significantly slower to approach its
asymptote than the compression function. This is what helps produce our
smoother gradients and hide the aggressiveness of our compression.</p>
smoother gradients and hide the aggressiveness of our compression. In some
sense I am replacing the lost brightness with desaturation, thus giving the
brain an alternate perceptual cue, which smoothly encodes several orders of
magnitude more brightness than is available in the output screen.</p>

<p>The complete shader code is quite small, with only three divides and
those only applied to colors over the 1:1 limit:<br/>
Expand Down Expand Up @@ -447,7 +459,7 @@ <h3 id="#commerce">Commerce tone mapper</h3>
<figure>
<model-viewer
id="demo"
src="https://cdn.glitch.global/93f14439-9407-4d48-8b56-aa0afd9b2886/MacbethBalls.glb?v=1704772991939"
src="../../shared-assets/models/MacbethBalls.glb"
tone-mapping="commerce"
ar
camera-controls
Expand Down Expand Up @@ -479,6 +491,50 @@ <h3 id="#commerce">Commerce tone mapper</h3>
<figcaption>Tone mapper test demo.</figcaption>
</figure>

<h3 id="#validation">Validation</h3>

<p>The best end-to-end validation we have for color accuracy is to apply an
unrealistic, analytic lighting environment: a white furnace test, where the
lighting is exactly uniform [1, 1, 1] white everywhere. This allows us to
expect a nearly-exact reproduction of baseColor to the output render, and
thus ensure our tone mapping function is not introducing further
changes.</p>

<p>Our 3D Macbeth chart model is ideal for this validation because tone
mapping is not applied at all to unlit materials, so the unlit spheres serve
as ground truth color comparisons for the PBR spheres. As you can see, they match
very well, in fact as close as is possible to match for PBR: there are
two expected sources of difference.</p>

<figure>
<model-viewer
id="demo"
src="../../shared-assets/models/MacbethBalls.glb"
skybox-image="../../shared-assets/environments/white_furnace.hdr"
tone-mapping="commerce"
ar
camera-controls
shadow-intensity="1"
>
</model-viewer>
<figcaption>Commerce tone mapper validation.</figcaption>
</figure>

<p>The first difference is from the Fresnel effect: on shiny materials, the
reflection loses material color near grazing angles. This is a physical
reality and causes the white halos on the edges of the shiny (back) spheres.
If you turn the model until the unlit spheres overlap the middle of the
shiny spheres, you'll see that the color match is exact at normal (center)
reflection.</p>

<p>The second effect is multi-scattering, which causes the dark-colored
matte (front) spheres to be slightly darker than their unlit comparisons.
This is also intentional, as matte materials are rough, thus forming
microscopic cavities that cause slight ambient occlusion and allow dark
materials more light bounces to absorb energy. Accurate PBR renderers
include this effect because a single material will in fact become brighter
as it is polished.</p>

<h3 id="#white">White point</h3>

<p>The tl;dr of this section is that you can safely skip it. It is a
Expand Down Expand Up @@ -583,21 +639,10 @@ <h3 id="#white">White point</h3>
updateToneMapper(toneMV, checkbox.checked ? "4" : "1");
});

const envMV = document.querySelector("#environments");
const envCycle = [
"../../shared-assets/environments/spruit_sunrise_1k_HDR.hdr",
"../../shared-assets/environments/whipple_creek_regional_park_04_1k.hdr",
"../../shared-assets/environments/lebombo_1k.hdr",
"../../shared-assets/environments/aircraft_workshop_01_1k.hdr",
"../../shared-assets/environments/music_hall_01_1k.hdr",
"../../shared-assets/environments/pillars_1k.hdr",
"../../shared-assets/environments/neutral.hdr"
];

setInterval(() => {
const cycleIndex = envCycle.indexOf(envMV.skyboxImage);
envMV.skyboxImage = envCycle[(cycleIndex + 1) % envCycle.length];
}, 3000);
const reachMV = document.querySelector("#reachable");
document.querySelector('#set').addEventListener("input", (event) => {
reachMV.src = event.target.value;
});
</script>
</body>
</html>

0 comments on commit 81b2f92

Please sign in to comment.