clean up of texture.encoding support of WebGLRenderTarget

[bhouston]

Dealing with @mrdoob's suggestion here: #8217 (comment)

[mrdoob]

Thanks!

[mrdoob]

In case you missed it... Any comments about this?
#8117 (comment)

[bhouston]

Well, I am not sure exactly what that is. There are a few different color profiles in browser space -- there is possibly an color profile stored in PNG and JPG and other files, often sRGB or Adobe RGB or even a LUT. If this applies that, it would be useful and means that one can get rid of sRGB decoding as long as one is loading the data directly from an HTMLImageElement. Although does it always apply the same profile to these images based on the image headers or does it use some type of default browser setting?

I guess some experiments are in order.

How does that option interact with https://www.khronos.org/registry/webgl/extensions/EXT_sRGB/ ?

I think that both situations can be handled via falling back to texture.decode = THREE.LinearEncoding at least in relation to the generation of the shader, thus letting non-glsl parts of the system handle the decoding of the texture?

I am currently adding support for output encoding. The primary reason for the encoding is for using HDR formats, RGBM, RGBE and RGBD, while using it for sRGB is sort of a side benefit because it works on every browser regardless of extensions or WebGL 2.

[mrdoob]

How does that option interact with https://www.khronos.org/registry/webgl/extensions/EXT_sRGB/ ?

/ping @kenrussell

[kenrussell]

Sorry for the delay replying.

I'm not an expert on color spaces or color profiles. My understanding is that EXT_sRGB provides two main benefits:

1. Allows you to use sRGB colorspace textures well. Sampling the texture in a shader performs the sRGB-to-linear conversion with high precision on the GPU.
2. Allows you to render to an sRGB format framebuffer, which provides more resolution in darker areas (i.e., less banding and more details in shadowed areas). Blending is done in linear space on the GPU by doing the sRGB-to-linear conversion per sample in high precision, doing the blending, and then doing the reverse conversion and quantization during rendering.

You all have much more experience than I do with Three's colorspace handling, and what's needed for HDR rendering. I can provide input on what some browsers do during image decoding. Please take the following with a grain of salt, since I might well be wrong on some of the details.

1. Chrome currently bakes the display's color profile into decoded images. This doesn't really conform to the CSS spec, which currently specifies that all colors on the web are defined in the sRGB color space, but does provide reasonable support for blending and large-gamut images. Work is underway to stop doing this and behave more like Safari and Edge, below.
2. Safari and Edge decode all non-large-gamut images into the sRGB color space. The conversion from sRGB to the display's color profile occurs in the window system compositor. This means that when an HTMLImageElement is uploaded to a WebGL texture, on these browsers, the values sampled will be different than on Chrome, for example -- but the end result on the screen should be similar, since the WebGL-rendered canvas should go through the same colorspace conversion on the way to the screen.
3. Setting the UNPACK_COLORSPACE_CONVERSION_WEBGL pixel unpack parameter to NONE will disable all colorspace conversion during image decoding and upload. (The preferred way to set this will be to use the colorspaceConversion ImageBitmap constructor argument, as soon as that API is widely supported, which is in the process of happening now.) Unfortunately, you won't know what colorspace the incoming image was in, since there's no way to query that using JavaScript APIs. You might have to re-download the image as a Blob and parse the header manually.
4. I'm not sure exactly what Firefox does with colorspace handling right now. I think this browser is moving toward proper colorspace handling, but that it's behind a command line (or about:config) flag right now.

Hope this information helps. Let me know if I can answer questions.