From a2dbf757686baf0001446c22773a4ff35fe8a965 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Tue, 16 Mar 2021 09:19:41 +0100 Subject: [PATCH 01/29] Initial commit for environment light extension This is a continuation of PR #1850 made into KHR extension and based on Khronos gltf repo (not forked) --- .../Khronos/KHR_lights_environment/README.md | 175 +++++++++++ .../figures/Cube_map.svg | 274 ++++++++++++++++++ .../schema/environment.schema.json | 46 +++ .../glTF.KHR_lights_environment.schema.json | 21 ++ .../scene.KHR_lights_environment.schema.json | 21 ++ 5 files changed, 537 insertions(+) create mode 100644 extensions/2.0/Khronos/KHR_lights_environment/README.md create mode 100644 extensions/2.0/Khronos/KHR_lights_environment/figures/Cube_map.svg create mode 100644 extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json create mode 100644 extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json create mode 100644 extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md new file mode 100644 index 0000000000..24d4c061d4 --- /dev/null +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -0,0 +1,175 @@ + +# KHR_lights_environment + +## Contributors + +* Norbert Nopper, +* Rickard Sahlin, +* Gary Hsu, Microsoft, +* Mike Bond, Adobe, + +## Status + +Draft + +## Dependencies + +Written against the glTF 2.0 spec. + +## Overview +This extension provides the ability to define image-based lights in a glTF scene. Image-based lights consist of an environment map that represents specular radiance for the scene as well as irradiance information. +This can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the IBL needs to be distributed. +It can also be used together with model data, for usecases where a model shall be displayed in a defined environment. + +Many 3D tools and engines support image-based global illumination but the exact technique and data formats employed vary. Using this extension, tools can export and engines can import image-based lights and the result should be highly consistent. + +This extension specifies exactly one way to format and reference the environment map to be used. The goals of this are two-fold. First, it makes implementing support for this extension easier. Secondly, it ensures that rendering of the image-based lighting is consistent across runtimes. + +A conforming implementation of this extension must be able to load the image-based environment data and render the PBR materials using this lighting. + +This extension is based on EXT_lights_image_based. + +The environment light is defined by a cubemap. +Cubemaps can be supplied pre-filtered, by including the pre-filtered mip-levels, otherwise filtering will be performed at load time. +If a compressed texture format is used then pre-filtered mip-levels must be included. + +How to pre-filter and/or sample the panorma image is implicitly definded by the [materials BRDF](https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#appendix-b-brdf-implementation). + + +## Declaring an environment light + +The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. +Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and rotation and intensity values. +The cubemap is defined by texture references, this means that the source format for textures can be extended by using an extension such as KHR_texture_basisu. +If image source contains a cubemap pyramid it must be pre-filtered and contain the needed mip-levels see Specular Radiance Cubemaps. + +```json +"extensions": { + "KHR_lights_environment" : { + "lights": [ + { + "intensity": 1.0, + "rotation": [0, 0, 0, 1], + "irradianceCoefficients": [...3 x 9 array of floats...], + "specularTextures": [ + [... 6 cube faces for mip 0 ...] + ], + } + ] + } +} +``` + + +### Defining the environment light + +One or more environment lights cane be defined inside [glTF.schema.json](https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/schema/glTF.schema.json) + +```json +"extensions": { + "EXT_lights_environment": { + "environments": [ + { + "source": 0, + "intensity": 1.0, + "frontside": "+X" + } + ] + } +} +``` + +## Specular Radiance Cubemaps + +The cubemap used for specular radiance is defined as separate images for each cube face. +The mip levels shall evenly map to roughness values from 0 to 1 in the PBR material and should be generated with a principled multi-scatter GGX normal distribution. The data in the maps represents illuminance in candela per square meter (nit). + +The entire mip chain of images should not be generated. +Instead, the lowest-resolution mip should have sufficient size to represent the maximally-blurred radiance map (say, 16x16) corresponding to roughness=1. The `specularImageSize` value defines the largest dimension of mip 0 and, taken together with the number of defined mips, should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. + +Cube faces are defined in the following order and adhere to the standard orientations as shown in the image bellow. +1. Positive X +1. Negative X +1. Positive Y +1. Negative Y +1. Positive Z +1. Negative Z + +
+ +
Cube map orientation reference.
Image by Microwerx - Own work, CC BY-SA 4.0, Link
+
+ +Note that for this extension, each saved image must be flipped about its vertical axis to correspond with the way glTF references texture space. + +https://en.wikipedia.org/wiki/Cube_mapping + + + +## Irradiance Coefficients + +This extension uses spherical harmonic coefficients to define irradiance used for diffuse lighting. Coefficients are calculated for the first 3 SH bands (l=2) and take the form of a 9x3 array. +[Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) +[An Efficient Representation for Irradiance Environment Maps](http://graphics.stanford.edu/papers/envmap/) + +### Using the environment light + +The environment light is utilized by a scene. + +```json +"scenes": [ + { + "nodes": [ + 0 + ], + "extensions": { + "EXT_lights_environment": { + "environment": 0 + } + } + } + ] +``` + +## Adding Light Instances to Scenes + +Each scene can have a single IBL light attached to it by defining the `extensions.EXT_lights_image_based` property and, within that, an index into the `lights` array using the `light` property. + +```javascript +"scenes" : [ + { + "extensions" : { + "EXT_lights_image_based" : { + "light" : 0 + } + } + } +] +``` + +### Image-Based Light Properties + +| Property | Description | Required | +|:-----------------------|:------------------------------------------| :--------------------------| +| `name` | Name of the light. | No | +| `rotation` | Quaternion that represents the rotation of the IBL environment. | No, Default: `[0.0, 0.0, 0.0, 1.0]` | +| `intensity` | Brightness multiplier for environment. | No, Default: `1.0` | +| `irradianceCoefficients` | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | :white_check_mark: Yes | +| `specularTextures` | Declares an array of the cubemap textures, this array must contain 6 tetures. | :white_check_mark: Yes | + + + + +## glTF Schema Updates + +- [glTF.KHR_lights_environment.schema.json](schema/glTF.KHR_lights_environment.schema.json) +- [environment.schema.json](schema/environment.schema.json) +- [scene.KHR_lights_environment.schema.json](schema/scene.KHR_lights_environment.schema.json) + +## Known Implementations + +* `TODO: Add implementations` + +## Reference + +* `TODO: Add references` \ No newline at end of file diff --git a/extensions/2.0/Khronos/KHR_lights_environment/figures/Cube_map.svg b/extensions/2.0/Khronos/KHR_lights_environment/figures/Cube_map.svg new file mode 100644 index 0000000000..81a1c5414c --- /dev/null +++ b/extensions/2.0/Khronos/KHR_lights_environment/figures/Cube_map.svg @@ -0,0 +1,274 @@ + + + + + + + + + + + + + + + + + -X FACE 1 + + -XFACE 1 + + + +Z FACE 4 + + +ZFACE 4 + + + +X FACE 0 + + +XFACE 0 + + + -Z FACE 5 + + -ZFACE 5 + + + +Y FACE 2 + + +YFACE 2 + + + -Y FACE 3 + + -YFACE 3 + + + + v + + v + + + u + + u + + + + + + + + + + + v + + v + + + u + + u + + + + + + + + + + + v + + v + + + u + + u + + + + + + + + + + + v + + v + + + u + + u + + + + + + + + + + + v + + v + + + u + + u + + + + + + + + + + + v + + v + + + u + + u + + + + + + + + + + + y + + y + + + z + + z + + + + + + + + + + + y + + y + + + x + + x + + + + + + + + + + + y + + y + + + -z + + -z + + + + + + + + + + + y + + y + + + -x + + -x + + + + + + + + + + + z + + z + + + x + + x + + + + + + + + + + + -z + + -z + + + x + + x + + + + + + + + + + diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json new file mode 100644 index 0000000000..c85b087e6c --- /dev/null +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json @@ -0,0 +1,46 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema", + "title": "EXT_lights_environment glTF extension", + "type": "object", + "allOf": [ { "$ref": "glTFChildOfRootProperty.schema.json" } ], + "properties": { + "source": { + "allOf": [ { "$ref": "glTFid.schema.json" } ], + "description": "The index of the image used by this environment. Color information is in linear space." + }, + "intensity": { + "type": "number", + "description": "Intensity of the light source in linear space. Luminance expressed as Candela per meter squared (cd/m2).", + "default": 1.0, + "minimum": 0 + }, + "frontside": { + "type": "string", + "description": "Specifies the front side. +Y and -Y are excluded by purpose.", + "default": "+X", + "anyOf": [ + { + "enum": [ "+X" ] + }, + { + "enum": [ "-X" ] + }, + { + "enum": [ "+Z" ] + }, + { + "enum": [ "-Z" ] + }, + { + "type": "string" + } + ] + }, + "name": { }, + "extensions": { }, + "extras": { } + }, + "required": [ + "source" + ] +} \ No newline at end of file diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json new file mode 100644 index 0000000000..63ad1ee850 --- /dev/null +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json @@ -0,0 +1,21 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema", + "title": "EXT_lights_environment glTF extension", + "type": "object", + "allOf": [ { "$ref": "glTFProperty.schema.json" } ], + "properties": { + "environments": { + "type": "array", + "items": { + "type": "object", + "$ref": "environment.schema.json" + }, + "minItems": 1 + }, + "extensions": {}, + "extras": {} + }, + "required": [ + "environments" + ] +} \ No newline at end of file diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json new file mode 100644 index 0000000000..1557ae7216 --- /dev/null +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json @@ -0,0 +1,21 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema", + "title": "EXT_lights_environment glTF extension", + "type": "object", + "allOf": [ { "$ref": "glTFProperty.schema.json" } ], + "properties": { + "environment": { + "allOf": [ + { + "$ref": "glTFid.schema.json" + } + ], + "description": "The id of the environment used for lighting referenced by this scene." + }, + "extensions": {}, + "extras": {} + }, + "required":[ + "environment" + ] +} \ No newline at end of file From 5e7ec02b839386881ba2daea15207b4cfb06d971 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Tue, 16 Mar 2021 09:32:13 +0100 Subject: [PATCH 02/29] Update schema --- .../schema/environment.schema.json | 46 -------------- .../glTF.KHR_lights_environment.schema.json | 14 ++--- .../schema/light.schema.json | 62 +++++++++++++++++++ .../scene.KHR_lights_environment.schema.json | 12 ++-- 4 files changed, 75 insertions(+), 59 deletions(-) delete mode 100644 extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json create mode 100644 extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json deleted file mode 100644 index c85b087e6c..0000000000 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/environment.schema.json +++ /dev/null @@ -1,46 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-04/schema", - "title": "EXT_lights_environment glTF extension", - "type": "object", - "allOf": [ { "$ref": "glTFChildOfRootProperty.schema.json" } ], - "properties": { - "source": { - "allOf": [ { "$ref": "glTFid.schema.json" } ], - "description": "The index of the image used by this environment. Color information is in linear space." - }, - "intensity": { - "type": "number", - "description": "Intensity of the light source in linear space. Luminance expressed as Candela per meter squared (cd/m2).", - "default": 1.0, - "minimum": 0 - }, - "frontside": { - "type": "string", - "description": "Specifies the front side. +Y and -Y are excluded by purpose.", - "default": "+X", - "anyOf": [ - { - "enum": [ "+X" ] - }, - { - "enum": [ "-X" ] - }, - { - "enum": [ "+Z" ] - }, - { - "enum": [ "-Z" ] - }, - { - "type": "string" - } - ] - }, - "name": { }, - "extensions": { }, - "extras": { } - }, - "required": [ - "source" - ] -} \ No newline at end of file diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json index 63ad1ee850..20fb95fc24 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json @@ -1,21 +1,21 @@ { "$schema": "http://json-schema.org/draft-04/schema", - "title": "EXT_lights_environment glTF extension", + "title": "KHR_lights_environment glTF extension", "type": "object", "allOf": [ { "$ref": "glTFProperty.schema.json" } ], "properties": { - "environments": { + "lights": { "type": "array", "items": { "type": "object", - "$ref": "environment.schema.json" + "$ref": "light.schema.json" }, "minItems": 1 }, - "extensions": {}, - "extras": {} + "extensions": { }, + "extras": { } }, - "required": [ - "environments" + "required":[ + "lights" ] } \ No newline at end of file diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json new file mode 100644 index 0000000000..44b406e445 --- /dev/null +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json @@ -0,0 +1,62 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema", + "title": "light", + "type": "object", + "description": "An image based environment light.", + "allOf" : [ { "$ref" : "glTFChildOfRootProperty.schema.json" } ], + "properties": { + "rotation": { + "type": "array", + "items": { + "type": "number", + "minimum": -1.0, + "maximum": 1.0 + }, + "minItems": 4, + "maxItems": 4, + "description": "Quaternion that represents the rotation of the IBL environment.", + "default": [0, 0, 0, 1] + }, + "intensity": { + "type": "number", + "description": "Brightness multiplier for environment.", + "default": 1.0, + "minimum": 0.0 + }, + "irradianceCoefficients": { + "description": "Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array.", + "type": "array", + "items": { + "type": "array", + "items": { + "type": "number" + }, + "minItems": 3, + "maxItems": 3 + }, + "minItems": 9, + "maxItems": 9 + }, + "specularTextures": { + "description": "Declares an array of the cubemap textures, one texture for each cube face. This array must contain 6 references.", + "type": "array", + "items": { + "type": "array", + "items": { + "type": "integer", + "minimum": 0 + }, + "minItems": 6, + "maxItems": 6 + }, + "minItems": 1 + }, + "name": { }, + "extensions": { }, + "extras": { } + }, + "required": [ + "irradianceCoefficients", + "specularTextures", + ] +} diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json index 1557ae7216..5a2272e145 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/scene.KHR_lights_environment.schema.json @@ -1,21 +1,21 @@ { "$schema": "http://json-schema.org/draft-04/schema", - "title": "EXT_lights_environment glTF extension", + "title": "KHR_lights_environment scene extension", "type": "object", "allOf": [ { "$ref": "glTFProperty.schema.json" } ], "properties": { - "environment": { + "light": { "allOf": [ { "$ref": "glTFid.schema.json" } ], - "description": "The id of the environment used for lighting referenced by this scene." + "description": "The id of the light referenced by this scene." }, - "extensions": {}, - "extras": {} + "extensions": { }, + "extras": { } }, "required":[ - "environment" + "light" ] } \ No newline at end of file From e44c245dd9504b6cfb706f5f3491f7f8098da2fb Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Tue, 16 Mar 2021 15:22:44 +0100 Subject: [PATCH 03/29] Simplify by using KTX2 for cubemaps --- .../Khronos/KHR_lights_environment/README.md | 98 ++++++------------- 1 file changed, 32 insertions(+), 66 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 24d4c061d4..a5331bbb6b 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -17,30 +17,34 @@ Draft Written against the glTF 2.0 spec. ## Overview -This extension provides the ability to define image-based lights in a glTF scene. Image-based lights consist of an environment map that represents specular radiance for the scene as well as irradiance information. -This can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the IBL needs to be distributed. -It can also be used together with model data, for usecases where a model shall be displayed in a defined environment. +This extension provides the ability to define image-based lights in a glTF scene. +Image-based lights consist of an environment map that represents specular radiance for the scene as well as irradiance information. +This can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the IBL needs to be distributed. +It can also be used together with model data, for usecases where a model shall be displayed in a defined environment. -Many 3D tools and engines support image-based global illumination but the exact technique and data formats employed vary. Using this extension, tools can export and engines can import image-based lights and the result should be highly consistent. +Many 3D tools and engines support image-based global illumination but the exact technique and data formats employed vary. +Using this extension, tools can export and engines can import image-based lights and the result should be highly consistent. -This extension specifies exactly one way to format and reference the environment map to be used. The goals of this are two-fold. First, it makes implementing support for this extension easier. Secondly, it ensures that rendering of the image-based lighting is consistent across runtimes. +This extension specifies exactly one way to format and reference the environment map to be used. +The goals of this are two-fold. +First, it makes implementing support for this extension easier. +Secondly, it ensures that rendering of the image-based lighting is consistent across runtimes. -A conforming implementation of this extension must be able to load the image-based environment data and render the PBR materials using this lighting. +A conforming implementation of this extension must be able to load the image-based environment data and render the PBR materials using this lighting. -This extension is based on EXT_lights_image_based. - -The environment light is defined by a cubemap. -Cubemaps can be supplied pre-filtered, by including the pre-filtered mip-levels, otherwise filtering will be performed at load time. -If a compressed texture format is used then pre-filtered mip-levels must be included. - -How to pre-filter and/or sample the panorma image is implicitly definded by the [materials BRDF](https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#appendix-b-brdf-implementation). +This extension is based on EXT_lights_image_based, with added support for KTX2 as a carrier for cubemap and HDR texture formats. +The environment light is defined by a cubemap. +Cubemaps can be supplied pre-filtered, by including the pre-filtered mip-levels, otherwise filtering will be performed at load time. +[TBD - point to relevant filtering algorithm] +If a compressed texture format is used then pre-filtered mip-levels must be included. ## Declaring an environment light -The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. -Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and rotation and intensity values. -The cubemap is defined by texture references, this means that the source format for textures can be extended by using an extension such as KHR_texture_basisu. +The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. +Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and intensity values. +The cubemap is defined by texture references to KTX2 files containing a cubemap, these files can contain compressed textures using KHR_texture_basisu + If image source contains a cubemap pyramid it must be pre-filtered and contain the needed mip-levels see Specular Radiance Cubemaps. ```json @@ -49,72 +53,35 @@ If image source contains a cubemap pyramid it must be pre-filtered and contain t "lights": [ { "intensity": 1.0, - "rotation": [0, 0, 0, 1], "irradianceCoefficients": [...3 x 9 array of floats...], - "specularTextures": [ - [... 6 cube faces for mip 0 ...] - ], + "specularCubemaps": [... Reference to KTX2 cubemaps...], } ] } } ``` - -### Defining the environment light - -One or more environment lights cane be defined inside [glTF.schema.json](https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/schema/glTF.schema.json) - -```json -"extensions": { - "EXT_lights_environment": { - "environments": [ - { - "source": 0, - "intensity": 1.0, - "frontside": "+X" - } - ] - } -} -``` - ## Specular Radiance Cubemaps The cubemap used for specular radiance is defined as separate images for each cube face. The mip levels shall evenly map to roughness values from 0 to 1 in the PBR material and should be generated with a principled multi-scatter GGX normal distribution. The data in the maps represents illuminance in candela per square meter (nit). -The entire mip chain of images should not be generated. -Instead, the lowest-resolution mip should have sufficient size to represent the maximally-blurred radiance map (say, 16x16) corresponding to roughness=1. The `specularImageSize` value defines the largest dimension of mip 0 and, taken together with the number of defined mips, should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. - -Cube faces are defined in the following order and adhere to the standard orientations as shown in the image bellow. -1. Positive X -1. Negative X -1. Positive Y -1. Negative Y -1. Positive Z -1. Negative Z - -
- -
Cube map orientation reference.
Image by Microwerx - Own work, CC BY-SA 4.0, Link
-
- -Note that for this extension, each saved image must be flipped about its vertical axis to correspond with the way glTF references texture space. - -https://en.wikipedia.org/wiki/Cube_mapping +If mip-levels are not included, then the entire mip chain of images should not be generated. +Instead, the lowest-resolution mip should have sufficient size to represent the maximally-blurred radiance map (say, 16x16) corresponding to roughness=1. The texture references defines the largest dimension of mip 0 and, taken together with the number of defined mips, should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. +Cube faces are defined in the KTX2 format specification and this extension adheres this. ## Irradiance Coefficients -This extension uses spherical harmonic coefficients to define irradiance used for diffuse lighting. Coefficients are calculated for the first 3 SH bands (l=2) and take the form of a 9x3 array. +This extension uses spherical harmonic coefficients to define irradiance used for diffuse lighting. +Coefficients are calculated for the first 3 SH bands (l=2) and take the form of a 9x3 array. [Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) [An Efficient Representation for Irradiance Environment Maps](http://graphics.stanford.edu/papers/envmap/) ### Using the environment light -The environment light is utilized by a scene. +The environment light is utilized by a scene. ```json "scenes": [ @@ -123,7 +90,7 @@ The environment light is utilized by a scene. 0 ], "extensions": { - "EXT_lights_environment": { + "KHR_lights_environment": { "environment": 0 } } @@ -133,13 +100,13 @@ The environment light is utilized by a scene. ## Adding Light Instances to Scenes -Each scene can have a single IBL light attached to it by defining the `extensions.EXT_lights_image_based` property and, within that, an index into the `lights` array using the `light` property. +Each scene can have a single IBL light attached to it by defining the `extensions.KHR_lights_image_based` property and, within that, an index into the `lights` array using the `light` property. ```javascript "scenes" : [ { "extensions" : { - "EXT_lights_image_based" : { + "KHR_lights_image_based" : { "light" : 0 } } @@ -152,10 +119,9 @@ Each scene can have a single IBL light attached to it by defining the `extension | Property | Description | Required | |:-----------------------|:------------------------------------------| :--------------------------| | `name` | Name of the light. | No | -| `rotation` | Quaternion that represents the rotation of the IBL environment. | No, Default: `[0.0, 0.0, 0.0, 1.0]` | | `intensity` | Brightness multiplier for environment. | No, Default: `1.0` | | `irradianceCoefficients` | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | :white_check_mark: Yes | -| `specularTextures` | Declares an array of the cubemap textures, this array must contain 6 tetures. | :white_check_mark: Yes | +| `specularCubemaps` | Cubemap texture | :white_check_mark: Yes | @@ -172,4 +138,4 @@ Each scene can have a single IBL light attached to it by defining the `extension ## Reference -* `TODO: Add references` \ No newline at end of file +* `TODO: Add references` From 2b2a17420d15cd023d4011ef2b395552298610ef Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Tue, 16 Mar 2021 15:26:50 +0100 Subject: [PATCH 04/29] Remove rotation and update to use specularCubemaps --- .../schema/light.schema.json | 22 +++++-------------- 1 file changed, 5 insertions(+), 17 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json index 44b406e445..875b14b0f5 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json @@ -5,18 +5,6 @@ "description": "An image based environment light.", "allOf" : [ { "$ref" : "glTFChildOfRootProperty.schema.json" } ], "properties": { - "rotation": { - "type": "array", - "items": { - "type": "number", - "minimum": -1.0, - "maximum": 1.0 - }, - "minItems": 4, - "maxItems": 4, - "description": "Quaternion that represents the rotation of the IBL environment.", - "default": [0, 0, 0, 1] - }, "intensity": { "type": "number", "description": "Brightness multiplier for environment.", @@ -37,8 +25,8 @@ "minItems": 9, "maxItems": 9 }, - "specularTextures": { - "description": "Declares an array of the cubemap textures, one texture for each cube face. This array must contain 6 references.", + "specularCubemaps": { + "description": "Declares an array of the cubemap textures, by using KTX2. Each entry must point to a KTX2 file with a cubemap containing 6 sides.", "type": "array", "items": { "type": "array", @@ -46,8 +34,8 @@ "type": "integer", "minimum": 0 }, - "minItems": 6, - "maxItems": 6 + "minItems": 1, + "maxItems": 1 }, "minItems": 1 }, @@ -57,6 +45,6 @@ }, "required": [ "irradianceCoefficients", - "specularTextures", + "specularCubemaps", ] } From 0370801ce90c15eaec951bffc53508c01cf34224 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Tue, 16 Mar 2021 15:37:16 +0100 Subject: [PATCH 05/29] Add KTX2 float formats --- extensions/2.0/Khronos/KHR_lights_environment/README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index a5331bbb6b..621ea31a93 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -43,7 +43,11 @@ If a compressed texture format is used then pre-filtered mip-levels must be incl The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and intensity values. -The cubemap is defined by texture references to KTX2 files containing a cubemap, these files can contain compressed textures using KHR_texture_basisu +The cubemap is defined by texture references to KTX2 files containing a cubemap, these files can contain compressed textures using KHR_texture_basisu. +KTX2 files may use the following formats for HDR support: +VK_FORMAT_B10G11R11_UFLOAT_PACK32 +VK_FORMAT_E5B9G9R9_UFLOAT_PACK32 +VK_FORMAT_BC6H_UFLOAT_BLOCK If image source contains a cubemap pyramid it must be pre-filtered and contain the needed mip-levels see Specular Radiance Cubemaps. From c2df7bbcd8e32f1b350f67eb40e81dd44969702f Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Wed, 17 Mar 2021 16:29:20 +0100 Subject: [PATCH 06/29] Clarify use of other KHR extensions --- .../Khronos/KHR_lights_environment/README.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 621ea31a93..4ae4e4c418 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -15,6 +15,7 @@ Draft ## Dependencies Written against the glTF 2.0 spec. +This extension may use KHR_texture_basisu, KHR_texture_float_bc6h or KHR_texture_float_rgb9e5 ## Overview This extension provides the ability to define image-based lights in a glTF scene. @@ -32,9 +33,10 @@ Secondly, it ensures that rendering of the image-based lighting is consistent ac A conforming implementation of this extension must be able to load the image-based environment data and render the PBR materials using this lighting. -This extension is based on EXT_lights_image_based, with added support for KTX2 as a carrier for cubemap and HDR texture formats. +This extension is based on EXT_lights_image_based, with added support for KTX2 as a carrier for [cubemap and] HDR texture formats. The environment light is defined by a cubemap. +[TBD - shall a separate KHR_texture_cubemap extension be created?] Cubemaps can be supplied pre-filtered, by including the pre-filtered mip-levels, otherwise filtering will be performed at load time. [TBD - point to relevant filtering algorithm] If a compressed texture format is used then pre-filtered mip-levels must be included. @@ -43,13 +45,9 @@ If a compressed texture format is used then pre-filtered mip-levels must be incl The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and intensity values. -The cubemap is defined by texture references to KTX2 files containing a cubemap, these files can contain compressed textures using KHR_texture_basisu. -KTX2 files may use the following formats for HDR support: -VK_FORMAT_B10G11R11_UFLOAT_PACK32 -VK_FORMAT_E5B9G9R9_UFLOAT_PACK32 -VK_FORMAT_BC6H_UFLOAT_BLOCK +The cubemap is defined by texture references to a KTX2 file containing a cubemap, these files can contain compressed textures using KHR_texture_basisu +or use a float texture format as defined by KHR_texture_float_bc6h or KHR_texture_float_rgb95e. -If image source contains a cubemap pyramid it must be pre-filtered and contain the needed mip-levels see Specular Radiance Cubemaps. ```json "extensions": { @@ -67,13 +65,13 @@ If image source contains a cubemap pyramid it must be pre-filtered and contain t ## Specular Radiance Cubemaps -The cubemap used for specular radiance is defined as separate images for each cube face. +The cubemap used for specular radiance is defined as a cubemap containing separate images for each cube face. The mip levels shall evenly map to roughness values from 0 to 1 in the PBR material and should be generated with a principled multi-scatter GGX normal distribution. The data in the maps represents illuminance in candela per square meter (nit). -If mip-levels are not included, then the entire mip chain of images should not be generated. +If mip-levels are not included, then the entire mip chain of images should not be generated. Instead, the lowest-resolution mip should have sufficient size to represent the maximally-blurred radiance map (say, 16x16) corresponding to roughness=1. The texture references defines the largest dimension of mip 0 and, taken together with the number of defined mips, should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. -Cube faces are defined in the KTX2 format specification and this extension adheres this. +Cube faces are defined in the KTX2 format specification, users of this extension must follow the KTX2 cubemap specification. ## Irradiance Coefficients From 6612a008a6c5aff55f14f9de306f3a82ff752b83 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Wed, 7 Apr 2021 12:25:40 +0200 Subject: [PATCH 07/29] Updates according to review comments Clarification of cubemap generation and use of spherical harmonics. Remove duplicate section. Added images and textures to json example. Added implementation note for specular and irradiance sections. Updated contributors. --- .../Khronos/KHR_lights_environment/README.md | 99 ++++++++++++------- 1 file changed, 65 insertions(+), 34 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 4ae4e4c418..f6e3824fa5 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -7,6 +7,7 @@ * Rickard Sahlin, * Gary Hsu, Microsoft, * Mike Bond, Adobe, +* Ben Houston, ThreeKit, ## Status @@ -15,7 +16,7 @@ Draft ## Dependencies Written against the glTF 2.0 spec. -This extension may use KHR_texture_basisu, KHR_texture_float_bc6h or KHR_texture_float_rgb9e5 +This extension may use KHR_texture_basisu, KHR_texture_float_bc6h or KHR_texture_float_f16 ## Overview This extension provides the ability to define image-based lights in a glTF scene. @@ -26,37 +27,62 @@ It can also be used together with model data, for usecases where a model shall b Many 3D tools and engines support image-based global illumination but the exact technique and data formats employed vary. Using this extension, tools can export and engines can import image-based lights and the result should be highly consistent. -This extension specifies exactly one way to format and reference the environment map to be used. -The goals of this are two-fold. +This extension specifies exactly one way to format and reference the environment map to be used, the goals of this are two-fold. First, it makes implementing support for this extension easier. Secondly, it ensures that rendering of the image-based lighting is consistent across runtimes. A conforming implementation of this extension must be able to load the image-based environment data and render the PBR materials using this lighting. -This extension is based on EXT_lights_image_based, with added support for KTX2 as a carrier for [cubemap and] HDR texture formats. - -The environment light is defined by a cubemap. -[TBD - shall a separate KHR_texture_cubemap extension be created?] -Cubemaps can be supplied pre-filtered, by including the pre-filtered mip-levels, otherwise filtering will be performed at load time. -[TBD - point to relevant filtering algorithm] +The environment light is defined by a cubemap, this is to simplify realtime implementations as these are likely to use cubemap texture format. +Cubemaps can be supplied pre-filtered, by including the pre-filtered mip-levels, where each mip-level corresponds to increasing roughness values. +If cubemap does not include mip-levels then filtering will be performed at load time. If a compressed texture format is used then pre-filtered mip-levels must be included. ## Declaring an environment light The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and intensity values. -The cubemap is defined by texture references to a KTX2 file containing a cubemap, these files can contain compressed textures using KHR_texture_basisu -or use a float texture format as defined by KHR_texture_float_bc6h or KHR_texture_float_rgb95e. +The cubemap is defined by texture references to a KTX2 file containing a cubemap. +These files can contain compressed textures using KHR_texture_basisu or use a float texture format as defined by KHR_texture_float_bc6h or KHR_texture_float_f16 + +When the extension is used, it's mandated to use value image/ktx2 for the mimeType property of (cubemap) images that are referenced by the `specularCubemap` property of KHR_lights_environment extension object. + +The following will load the environment light using KHR_texture_basisu on clients that supports that extension, otherwise fall back to using KTX2 using VK_FORMAT_R8G8B8_SRGB. + +TODO: Clarify what KTX2 formats, if any, are supported by default. +Should a separate extension be created to enable ktx2 as image source? ```json +"textures": [ + { + "source": 0, + "extensions": { + "KHR_texture_basisu": { + "source": 1 + } + } + } +], +"images": [ + { + "name": "environment cubemap 0", + "uri": "cubemap0.ktx2", + "mimeType": "image/ktx2" + }, + { + "name": "environment cubemap basisu", + "uri": "cubemap_basisu.ktx2", + "mimeType": "image/ktx2" + } +], "extensions": { "KHR_lights_environment" : { "lights": [ { "intensity": 1.0, "irradianceCoefficients": [...3 x 9 array of floats...], - "specularCubemaps": [... Reference to KTX2 cubemaps...], + "specularCubemaps": [0], } ] } @@ -69,21 +95,41 @@ The cubemap used for specular radiance is defined as a cubemap containing separa The mip levels shall evenly map to roughness values from 0 to 1 in the PBR material and should be generated with a principled multi-scatter GGX normal distribution. The data in the maps represents illuminance in candela per square meter (nit). If mip-levels are not included, then the entire mip chain of images should not be generated. -Instead, the lowest-resolution mip should have sufficient size to represent the maximally-blurred radiance map (say, 16x16) corresponding to roughness=1. The texture references defines the largest dimension of mip 0 and, taken together with the number of defined mips, should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. +Instead, the lowest-resolution mip should have sufficient size to represent the maximally-blurred radiance map corresponding to roughness=1. +The texture references defines the largest dimension of mip 0 and should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. Cube faces are defined in the KTX2 format specification, users of this extension must follow the KTX2 cubemap specification. +## Implementation Note +This section is non-normative + +The cubemap textures are used both as direct light contribution and as source for reflection. +A performant way of achieving this is to store the reflection at different roughness values in mip-levels, +where mip-level 0 is for a mirror like reflection (roughness = 0) and the last mip-level is for maximum roughness (roughness = 0). +It is recommended to map roughness = 1 to a mip-level size that is larger than 1 * 1 pixels to avoid blockiness, it is generally enough to stop at 16 * 16 pixels. +To avoid oversampling of flat surfaces (low roughness values) it is recommended to use a 3D cubemap when possible. + +[Runtime filtering of mip-levels](https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-20-gpu-based-importance-sampling) + ## Irradiance Coefficients -This extension uses spherical harmonic coefficients to define irradiance used for diffuse lighting. +This extension may use spherical harmonic coefficients to define irradiance used for diffuse lighting. Coefficients are calculated for the first 3 SH bands (l=2) and take the form of a 9x3 array. -[Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) -[An Efficient Representation for Irradiance Environment Maps](http://graphics.stanford.edu/papers/envmap/) + +## Implementation Note +This section is non-normative + +Implementations may calculate the irradiance cubemap from the specular radiance cubemap and use instead. +One possible benefit with spherical harmonics is that it is generally enough evaluate the harmonics on a per vertex basis, instead of sampling a cubemap on a per fragment basis. + +[Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) +[An Efficient Representation for Irradiance Environment Maps](http://graphics.stanford.edu/papers/envmap/) ### Using the environment light The environment light is utilized by a scene. +Each scene can have a single environment light attached to it by defining the `extensions.KHR_lights_environment` property and, within that, an index into the `lights` array using the `light` property. ```json "scenes": [ @@ -93,30 +139,15 @@ The environment light is utilized by a scene. ], "extensions": { "KHR_lights_environment": { - "environment": 0 + "light": 0 } } } ] ``` -## Adding Light Instances to Scenes - -Each scene can have a single IBL light attached to it by defining the `extensions.KHR_lights_image_based` property and, within that, an index into the `lights` array using the `light` property. - -```javascript -"scenes" : [ - { - "extensions" : { - "KHR_lights_image_based" : { - "light" : 0 - } - } - } -] -``` -### Image-Based Light Properties +### Environment Light Properties | Property | Description | Required | |:-----------------------|:------------------------------------------| :--------------------------| @@ -140,4 +171,4 @@ Each scene can have a single IBL light attached to it by defining the `extension ## Reference -* `TODO: Add references` +[Irradiance Environment Maps](https://graphics.stanford.edu/papers/ravir_thesis/chapter4.pdf) From 3b3e9be4effffacb7c595bca1e0dc779edf2874b Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Thu, 8 Apr 2021 14:10:28 +0200 Subject: [PATCH 08/29] Updated wording related to KTX Elaborate on how KTX v2 is supported, add supported formats. --- .../2.0/Khronos/KHR_lights_environment/README.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index f6e3824fa5..c9586a4288 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -15,11 +15,11 @@ Draft ## Dependencies -Written against the glTF 2.0 spec. +Written against the glTF 2.0 spec. This extension may use KHR_texture_basisu, KHR_texture_float_bc6h or KHR_texture_float_f16 ## Overview -This extension provides the ability to define image-based lights in a glTF scene. +This extension provides the ability to define image-based lights in a glTF scene using KTX v2 images. Image-based lights consist of an environment map that represents specular radiance for the scene as well as irradiance information. This can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the IBL needs to be distributed. It can also be used together with model data, for usecases where a model shall be displayed in a defined environment. @@ -45,12 +45,15 @@ Each environment light definition consists of a single cubemap that describes th The cubemap is defined by texture references to a KTX2 file containing a cubemap. These files can contain compressed textures using KHR_texture_basisu or use a float texture format as defined by KHR_texture_float_bc6h or KHR_texture_float_f16 -When the extension is used, it's mandated to use value image/ktx2 for the mimeType property of (cubemap) images that are referenced by the `specularCubemap` property of KHR_lights_environment extension object. +When the extension is used, images shall use `image/ktx2` as mimeType for cubemaps that are referenced by the `specularCubemap` property of KHR_lights_environment extension object. +The texture type of the KTX v2 file shall be 'Cubemap' -The following will load the environment light using KHR_texture_basisu on clients that supports that extension, otherwise fall back to using KTX2 using VK_FORMAT_R8G8B8_SRGB. +Supported formats are: +VK_FORMAT_R8G8B8_SRGB +VK_FORMAT_R16G16B16_SFLOAT +VK_FORMAT_R32G32B32_SFLOAT -TODO: Clarify what KTX2 formats, if any, are supported by default. -Should a separate extension be created to enable ktx2 as image source? +The following will load the environment light using KHR_texture_basisu on clients that supports that extension, otherwise fall back to using KTX2 using VK_FORMAT_R8G8B8_SRGB. ```json From 905350af33af3e58d58b4a98c91db51ac1eb94a7 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Fri, 9 Apr 2021 12:38:11 +0200 Subject: [PATCH 09/29] Updates, reference KHR_texture_ktx Specify that this extension requires KHR_texture_ktx. Clarify that no pre-filtering shall be done for uncompressed formats. Add open issues. --- .../Khronos/KHR_lights_environment/README.md | 30 +++++++++++-------- 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index c9586a4288..0c8b5e1327 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -13,14 +13,23 @@ Draft +Open issues: +Should Spherical Harmonics be supported or should clients be forced to evaluate the specular cubemap for diffuse contribution. +The question comes down to if resolution supplied by 9 coefficients SH is enough for most usecases? +Since the SH are optional, and I believe the quality using 9 coefficients is reasonable, I am leaning towards keeping them. + +If compressed texture format is used how should pre-filtering for roughness reflection be done? +May lead to unpredicted results of clients shall decode then create mip-levels since the result will likely take much more GPU memory since compression is likely not supported on target. + ## Dependencies Written against the glTF 2.0 spec. -This extension may use KHR_texture_basisu, KHR_texture_float_bc6h or KHR_texture_float_f16 +This extension depends on KHR_texture_ktx +This extension may use KHR_texture_basisu or KHR_texture_float_bc6h to support compressed texture formats. ## Overview -This extension provides the ability to define image-based lights in a glTF scene using KTX v2 images. -Image-based lights consist of an environment map that represents specular radiance for the scene as well as irradiance information. +This extension provides the ability to define image-based lights in a glTF scene using KTX v2 images as defined by KHR_texture_ktx. +Image-based lights (environment map, light map) consist of a cubemap map that represents specular radiance for the scene, irradiance information and is the source for environment reflection. This can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the IBL needs to be distributed. It can also be used together with model data, for usecases where a model shall be displayed in a defined environment. @@ -34,24 +43,21 @@ Secondly, it ensures that rendering of the image-based lighting is consistent ac A conforming implementation of this extension must be able to load the image-based environment data and render the PBR materials using this lighting. The environment light is defined by a cubemap, this is to simplify realtime implementations as these are likely to use cubemap texture format. -Cubemaps can be supplied pre-filtered, by including the pre-filtered mip-levels, where each mip-level corresponds to increasing roughness values. -If cubemap does not include mip-levels then filtering will be performed at load time. -If a compressed texture format is used then pre-filtered mip-levels must be included. +Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > 0, pre-filtering shall be done by the client. +[See Specular radiance cubemaps](#specular-radiance-cubemaps) + +If a compressed texture format is used then pre-filtered mip-levels for roughness values > 0 shall be specified. ## Declaring an environment light The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and intensity values. The cubemap is defined by texture references to a KTX2 file containing a cubemap. -These files can contain compressed textures using KHR_texture_basisu or use a float texture format as defined by KHR_texture_float_bc6h or KHR_texture_float_f16 +These files can contain compressed textures using KHR_texture_basisu or compressed float texture format as defined by KHR_texture_float_bc6h. When the extension is used, images shall use `image/ktx2` as mimeType for cubemaps that are referenced by the `specularCubemap` property of KHR_lights_environment extension object. The texture type of the KTX v2 file shall be 'Cubemap' -Supported formats are: -VK_FORMAT_R8G8B8_SRGB -VK_FORMAT_R16G16B16_SFLOAT -VK_FORMAT_R32G32B32_SFLOAT The following will load the environment light using KHR_texture_basisu on clients that supports that extension, otherwise fall back to using KTX2 using VK_FORMAT_R8G8B8_SRGB. @@ -123,7 +129,7 @@ Coefficients are calculated for the first 3 SH bands (l=2) and take the form of ## Implementation Note This section is non-normative -Implementations may calculate the irradiance cubemap from the specular radiance cubemap and use instead. +Implementations may calculate the irradiance from the specular radiance cubemap and use instead. One possible benefit with spherical harmonics is that it is generally enough evaluate the harmonics on a per vertex basis, instead of sampling a cubemap on a per fragment basis. [Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) From 0df13f29ea675f738d90c54859816217081b70a8 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Fri, 9 Apr 2021 19:52:19 +0200 Subject: [PATCH 10/29] Update to open issues --- extensions/2.0/Khronos/KHR_lights_environment/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 0c8b5e1327..039ed54e2b 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -14,9 +14,10 @@ Draft Open issues: -Should Spherical Harmonics be supported or should clients be forced to evaluate the specular cubemap for diffuse contribution. -The question comes down to if resolution supplied by 9 coefficients SH is enough for most usecases? -Since the SH are optional, and I believe the quality using 9 coefficients is reasonable, I am leaning towards keeping them. +Should Spherical Harmonics be supported or should clients be forced to evaluate the specular cubemap for diffuse contribution? +The question comes down to if resolution supplied by 9 coefficients SH is enough for most usecases? +It could also be the case that a scene only wants diffuse contribution. +A link to an implementation that uses the cubemap for specular, diffuse and reflection would be beneficial. If compressed texture format is used how should pre-filtering for roughness reflection be done? May lead to unpredicted results of clients shall decode then create mip-levels since the result will likely take much more GPU memory since compression is likely not supported on target. From fdc5e5ba92ff8f5861cdd71cc182ea0d413d6e83 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Fri, 30 Apr 2021 09:15:49 +0200 Subject: [PATCH 11/29] Clarification for cubemap filtering --- .../Khronos/KHR_lights_environment/README.md | 36 ++++++++++++------- 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 039ed54e2b..d76690416f 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -25,7 +25,7 @@ May lead to unpredicted results of clients shall decode then create mip-levels s ## Dependencies Written against the glTF 2.0 spec. -This extension depends on KHR_texture_ktx +This extension depends on KHR_texture_ktx. This extension may use KHR_texture_basisu or KHR_texture_float_bc6h to support compressed texture formats. ## Overview @@ -49,27 +49,37 @@ Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > If a compressed texture format is used then pre-filtered mip-levels for roughness values > 0 shall be specified. + > **Implementation Note**: Implementations are free to ignore the pre-filtered mip-levels and generate the mip-levels for roughness values at runtime. + ## Declaring an environment light The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and intensity values. The cubemap is defined by texture references to a KTX2 file containing a cubemap. -These files can contain compressed textures using KHR_texture_basisu or compressed float texture format as defined by KHR_texture_float_bc6h. +These files may contain compressed textures using KHR_texture_basisu or compressed float texture format as defined by KHR_texture_float_bc6h. When the extension is used, images shall use `image/ktx2` as mimeType for cubemaps that are referenced by the `specularCubemap` property of KHR_lights_environment extension object. The texture type of the KTX v2 file shall be 'Cubemap' -The following will load the environment light using KHR_texture_basisu on clients that supports that extension, otherwise fall back to using KTX2 using VK_FORMAT_R8G8B8_SRGB. +The following will load the environment light using KHR_texture_ktx. -```json +```json + +"asset": { +"version": "2.0" +}, +"extensionsUsed": [ + "KHR_texture_ktx", + "KHR_lights_environment" +], "textures": [ { - "source": 0, "extensions": { - "KHR_texture_basisu": { - "source": 1 + "KHR_texture_ktx": { + "source": 0, + "layer": 0 } } } @@ -79,11 +89,6 @@ The following will load the environment light using KHR_texture_basisu on client "name": "environment cubemap 0", "uri": "cubemap0.ktx2", "mimeType": "image/ktx2" - }, - { - "name": "environment cubemap basisu", - "uri": "cubemap_basisu.ktx2", - "mimeType": "image/ktx2" } ], "extensions": { @@ -167,6 +172,10 @@ Each scene can have a single environment light attached to it by defining the `e | `specularCubemaps` | Cubemap texture | :white_check_mark: Yes | +## KTX v2 Images + +The texture type of the KTX v2 file shall be 'Cubemap' + ## glTF Schema Updates @@ -175,6 +184,9 @@ Each scene can have a single environment light attached to it by defining the `e - [environment.schema.json](schema/environment.schema.json) - [scene.KHR_lights_environment.schema.json](schema/scene.KHR_lights_environment.schema.json) + + + ## Known Implementations * `TODO: Add implementations` From 20d55caa84a720b75fbb62d47c32f73aa24232c3 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Fri, 30 Apr 2021 09:17:51 +0200 Subject: [PATCH 12/29] Clarification for cubemap filtering Add section for KTX v2 images, update json to include extensionsUsed From 70653feccf390809688b6887a50a68fedae0f23d Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Thu, 20 May 2021 09:51:39 +0200 Subject: [PATCH 13/29] Remove resolved issues --- extensions/2.0/Khronos/KHR_lights_environment/README.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index d76690416f..26e40180f0 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -13,15 +13,6 @@ Draft -Open issues: -Should Spherical Harmonics be supported or should clients be forced to evaluate the specular cubemap for diffuse contribution? -The question comes down to if resolution supplied by 9 coefficients SH is enough for most usecases? -It could also be the case that a scene only wants diffuse contribution. -A link to an implementation that uses the cubemap for specular, diffuse and reflection would be beneficial. - -If compressed texture format is used how should pre-filtering for roughness reflection be done? -May lead to unpredicted results of clients shall decode then create mip-levels since the result will likely take much more GPU memory since compression is likely not supported on target. - ## Dependencies Written against the glTF 2.0 spec. From 0f553cee478f323c713fe75a01990c710cdab668 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Wed, 26 May 2021 11:49:06 +0200 Subject: [PATCH 14/29] Updated light properties --- extensions/2.0/Khronos/KHR_lights_environment/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 26e40180f0..53f027e6d2 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -39,6 +39,7 @@ Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > [See Specular radiance cubemaps](#specular-radiance-cubemaps) If a compressed texture format is used then pre-filtered mip-levels for roughness values > 0 shall be specified. +This is to ensure deterministic texture memory usage. > **Implementation Note**: Implementations are free to ignore the pre-filtered mip-levels and generate the mip-levels for roughness values at runtime. @@ -159,8 +160,8 @@ Each scene can have a single environment light attached to it by defining the `e |:-----------------------|:------------------------------------------| :--------------------------| | `name` | Name of the light. | No | | `intensity` | Brightness multiplier for environment. | No, Default: `1.0` | -| `irradianceCoefficients` | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | :white_check_mark: Yes | -| `specularCubemaps` | Cubemap texture | :white_check_mark: Yes | +| `irradianceCoefficients` | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | +| `specularCubemaps` | Cubemap texture - this must be a KTX v2 image containing a cubemap | :white_check_mark: Yes | ## KTX v2 Images From 47063300a23a0a72dd6e04b1e5308773018741b2 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Wed, 26 May 2021 11:55:19 +0200 Subject: [PATCH 15/29] Update specularCubemap description --- extensions/2.0/Khronos/KHR_lights_environment/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 53f027e6d2..e5cd219190 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -161,7 +161,7 @@ Each scene can have a single environment light attached to it by defining the `e | `name` | Name of the light. | No | | `intensity` | Brightness multiplier for environment. | No, Default: `1.0` | | `irradianceCoefficients` | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | -| `specularCubemaps` | Cubemap texture - this must be a KTX v2 image containing a cubemap | :white_check_mark: Yes | +| `specularCubemaps` | Cubemap texture - this must be a KTX v2 image containing a cubemap. The texture source may use some other extension include additional texture format support. Although this is an array only one value is allowed. | :white_check_mark: Yes | ## KTX v2 Images From 3744d7eac94562b858805d3c91b626911c43ca67 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Wed, 26 May 2021 12:03:36 +0200 Subject: [PATCH 16/29] Moved `lights` into extension Instead of referring to an image declared in the source asset, this extension shall define the images referenced within the extension itself. This is in order to not include any data when this extension is not supported. --- .../2.0/Khronos/KHR_lights_environment/README.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index e5cd219190..bbd97afa9c 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -76,17 +76,19 @@ The following will load the environment light using KHR_texture_ktx. } } ], -"images": [ - { - "name": "environment cubemap 0", - "uri": "cubemap0.ktx2", - "mimeType": "image/ktx2" - } -], + "extensions": { "KHR_lights_environment" : { + "images": [ + { + "name": "environment cubemap 0", + "uri": "cubemap0.ktx2", + "mimeType": "image/ktx2" + } + ], "lights": [ { + "name": "environment light 0", "intensity": 1.0, "irradianceCoefficients": [...3 x 9 array of floats...], "specularCubemaps": [0], From 71d8f3aa482298f85a8585e95ffb33ed214c557c Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Fri, 15 Oct 2021 12:29:39 +0200 Subject: [PATCH 17/29] Updates to use local cubemap, clarifications Added property for boundingbox min and max and usage of local boundingbox. Clarification of texture and image usage. --- .../Khronos/KHR_lights_environment/README.md | 110 ++++++++++++++---- 1 file changed, 86 insertions(+), 24 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index bbd97afa9c..1d7b32baf7 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -39,18 +39,23 @@ Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > [See Specular radiance cubemaps](#specular-radiance-cubemaps) If a compressed texture format is used then pre-filtered mip-levels for roughness values > 0 shall be specified. +The number of specified mip-levels shall be log2 of the min value from texture width and height, ie a full mipmap pyramid. This is to ensure deterministic texture memory usage. +Cubemap images are declared as an array of images in the extension that is placed in the glTF root. + + > **Implementation Note**: Implementations are free to ignore the pre-filtered mip-levels and generate the mip-levels for roughness values at runtime. -## Declaring an environment light +## Declaring An Environment Light + +The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF, each scene can reference one these lights. +Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for irradiance, luminance factor and bounding box for localized cubemap. -The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF and then each scene can reference one. -Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance and intensity values. -The cubemap is defined by texture references to a KTX2 file containing a cubemap. -These files may contain compressed textures using KHR_texture_basisu or compressed float texture format as defined by KHR_texture_float_bc6h. +The cubemap is defined as an integer reference to one of the texture sources declared in the this extension. +These textures may use an extension to specify the format, for instance using KHR_texture_basisu or KHR_texture_float_bc6h. -When the extension is used, images shall use `image/ktx2` as mimeType for cubemaps that are referenced by the `specularCubemap` property of KHR_lights_environment extension object. +When the extension is used, images shall use `image/ktx2` as mimeType. The texture type of the KTX v2 file shall be 'Cubemap' @@ -85,13 +90,25 @@ The following will load the environment light using KHR_texture_ktx. "uri": "cubemap0.ktx2", "mimeType": "image/ktx2" } - ], + ], + "textures": [ + { + "extensions": { + "KHR_texture_ktx": { + "source": 0, + "layer": 0 + } + } + } + ], "lights": [ { "name": "environment light 0", - "intensity": 1.0, + "luminance": 1.0, "irradianceCoefficients": [...3 x 9 array of floats...], - "specularCubemaps": [0], + "boundingBoxMin": [-100, -100, -100], + "boundingBoxMax": [100, 100, 100], + "cubemap": 0, } ] } @@ -101,32 +118,60 @@ The following will load the environment light using KHR_texture_ktx. ## Specular Radiance Cubemaps The cubemap used for specular radiance is defined as a cubemap containing separate images for each cube face. -The mip levels shall evenly map to roughness values from 0 to 1 in the PBR material and should be generated with a principled multi-scatter GGX normal distribution. The data in the maps represents illuminance in candela per square meter (nit). +The mip levels shall evenly map to roughness values from 0 to 1 in the PBR material and should be generated with a principled multi-scatter GGX normal distribution. The data in the maps represents illuminance in candela per square meter. -If mip-levels are not included, then the entire mip chain of images should not be generated. -Instead, the lowest-resolution mip should have sufficient size to represent the maximally-blurred radiance map corresponding to roughness=1. -The texture references defines the largest dimension of mip 0 and should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. +If mip-levels are included they are considered to be an approximation of the reflection at varying roughness levels. + +Cube faces are defined in the KTX2 format specification, implementations of this extension must follow the KTX2 cubemap specification. + +The boundingbox for the cubemaps are declared, this means that the locality of the camera within the cubemap can be calculated. +Making it possible to calculate the localized reflection vector allowing for lighting and reflections that not always uses the centerpoint of the cubemap as a reference point. +[See Localized cubemaps](#localized-cubemaps) -Cube faces are defined in the KTX2 format specification, users of this extension must follow the KTX2 cubemap specification. -## Implementation Note +### Implementation Note This section is non-normative -The cubemap textures are used both as direct light contribution and as source for reflection. +If mip-levels are not included, then the entire mip chain of images does not have be generated. +Generally it is enough to generate mip-level such that the lowest-resolution mip have sufficient size to represent the maximally-blurred radiance map corresponding to roughness=1. +The texture references defines the largest dimension of mip 0 and should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. + + +In a realtime renderer the cubemap textures may be used as a source for radiance, irradiance and reflection. A performant way of achieving this is to store the reflection at different roughness values in mip-levels, where mip-level 0 is for a mirror like reflection (roughness = 0) and the last mip-level is for maximum roughness (roughness = 0). +The lod level is calculated like this: +lod = roughness * (numberOfMipLevels - 1) + It is recommended to map roughness = 1 to a mip-level size that is larger than 1 * 1 pixels to avoid blockiness, it is generally enough to stop at 16 * 16 pixels. To avoid oversampling of flat surfaces (low roughness values) it is recommended to use a 3D cubemap when possible. -[Runtime filtering of mip-levels](https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-20-gpu-based-importance-sampling) +[Runtime filtering of mip-levels](https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-20-gpu-based-importance-sampling) +[Creating prefiltered reflection map](https://docs.imgtec.com/Graphics_Techniques/PBR_with_IBL_for_PVR/topics/Assets/pbr_ibl__the_prefiltered_map.html) + +## Localized Cubemaps + +The cubemaps shall use a technique called localized cubemaps, also called local cubemaps or box projected cubemaps. +This introduces a boundingbox (min / max) that makes it possible to calculate a local corrected reflection vector. +Meaning that models displayed within the cubemap environment does not have to rely on the centerpoint for light and reflection lookup. +Instead it is possible to place objects and the camera within this environment. +The cubemap boundingbox is declared as a pair of 3D coordinates for min / max. + +### Implementaiton Note + +Here is an example of a realtime implementation of reflections using local cubemap: +[Implement reflections with a local cubemap](https://developer.arm.com/solutions/graphics-and-gaming/gaming-engine/unity/arm-guides-for-unity-developers/local-cubemap-rendering/implement-reflections-with-a-local-cubemap) + ## Irradiance Coefficients This extension may use spherical harmonic coefficients to define irradiance used for diffuse lighting. Coefficients are calculated for the first 3 SH bands (l=2) and take the form of a 9x3 array. +If irradiance coefficients are supplied the specular cubemaps shall not be used as a contribution to diffuse lighting calculations. +The occlusion parameter of a material will affect the contribution from the irradiance coefficients. -## Implementation Note +### Implementation Note This section is non-normative Implementations may calculate the irradiance from the specular radiance cubemap and use instead. @@ -158,12 +203,29 @@ Each scene can have a single environment light attached to it by defining the `e ### Environment Light Properties -| Property | Description | Required | -|:-----------------------|:------------------------------------------| :--------------------------| -| `name` | Name of the light. | No | -| `intensity` | Brightness multiplier for environment. | No, Default: `1.0` | -| `irradianceCoefficients` | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | -| `specularCubemaps` | Cubemap texture - this must be a KTX v2 image containing a cubemap. The texture source may use some other extension include additional texture format support. Although this is an array only one value is allowed. | :white_check_mark: Yes | +`textures` + +| Property | Type | Description | Required | +|:-----------------------|:-----------|:------------------------------------------| :--------------------------| +| `textures` | texture [1-* ] | Array of textures. | :white_check_mark: Yes | + + +`images` +| Property | Type | Description | Required | +|:-----------------------|:-----------|:------------------------------------------| :--------------------------| +| `images` | image [1-* ] | Array of images that reference KTX V 2 file containing at least one cubemap. | :white_check_mark: Yes | + + +`light` + +| Property | Type | Description | Required | +|:-----------------------|:-----------|:------------------------------------------| :--------------------------| +| `name` | String | Name of the light. | No | +| `luminanceFactor` | number | Luminance factor for the environment, in candela per square meter. | No, Default: `1.0` | +| `irradianceCoefficients` | number[9][3] | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | +| `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | :white_check_mark: Yes | +| `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | :white_check_mark: Yes | +| `cubemap` | integer | Reference to texture to be used as cubemap. The texture source may use some other extension to include additional texture format support. | :white_check_mark: Yes | ## KTX v2 Images From 73b3133f16bdb43bd283f7408403af94f42e2e56 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Fri, 15 Oct 2021 12:38:38 +0200 Subject: [PATCH 18/29] Add bbox min/max, use luminance and texture ref --- .../schema/light.schema.json | 35 +++++++++++-------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json index 875b14b0f5..ec3beec94c 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json @@ -5,9 +5,9 @@ "description": "An image based environment light.", "allOf" : [ { "$ref" : "glTFChildOfRootProperty.schema.json" } ], "properties": { - "intensity": { + "luminance": { "type": "number", - "description": "Brightness multiplier for environment.", + "description": " Luminance factor for the environment, in candela per square meter. .", "default": 1.0, "minimum": 0.0 }, @@ -25,26 +25,33 @@ "minItems": 9, "maxItems": 9 }, - "specularCubemaps": { - "description": "Declares an array of the cubemap textures, by using KTX2. Each entry must point to a KTX2 file with a cubemap containing 6 sides.", + "cubemap": { + "type": "integer", + "description": "Texture reference to one of the declared cubemap textures.", + }, + "boundingBoxMin": { + "description": "Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters)", "type": "array", - "items": { - "type": "array", "items": { - "type": "integer", - "minimum": 0 + "type": "number" }, - "minItems": 1, - "maxItems": 1 - }, - "minItems": 1 + "minItems": 3, + "maxItems": 3 + }, + "boundingBoxMax": { + "description": "Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters)", + "type": "array", + "items": { + "type": "number" + }, + "minItems": 3, + "maxItems": 3 }, "name": { }, "extensions": { }, "extras": { } }, "required": [ - "irradianceCoefficients", - "specularCubemaps", + "cubemap", ] } From 6f2b3fcf0cb700c7b8789086777d6af241c0b702 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Mon, 18 Oct 2021 11:43:27 +0200 Subject: [PATCH 19/29] Remove KHR_texture ktx extension An ibl is declared using a texture references, these are KHR_texture_ktx in order for other format specific texture extensions to be used with this textension. --- .../Khronos/KHR_lights_environment/README.md | 39 +++++++------------ 1 file changed, 14 insertions(+), 25 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 1d7b32baf7..118f97a7cf 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -71,19 +71,15 @@ The following will load the environment light using KHR_texture_ktx. "KHR_texture_ktx", "KHR_lights_environment" ], -"textures": [ - { - "extensions": { - "KHR_texture_ktx": { - "source": 0, - "layer": 0 - } - } - } -], "extensions": { "KHR_lights_environment" : { + "textures": [ + { + "source": 0, + "layer": 0 + } + ], "images": [ { "name": "environment cubemap 0", @@ -91,16 +87,6 @@ The following will load the environment light using KHR_texture_ktx. "mimeType": "image/ktx2" } ], - "textures": [ - { - "extensions": { - "KHR_texture_ktx": { - "source": 0, - "layer": 0 - } - } - } - ], "lights": [ { "name": "environment light 0", @@ -157,7 +143,7 @@ Meaning that models displayed within the cubemap environment does not have to re Instead it is possible to place objects and the camera within this environment. The cubemap boundingbox is declared as a pair of 3D coordinates for min / max. -### Implementaiton Note +### Implementation Note Here is an example of a realtime implementation of reflections using local cubemap: [Implement reflections with a local cubemap](https://developer.arm.com/solutions/graphics-and-gaming/gaming-engine/unity/arm-guides-for-unity-developers/local-cubemap-rendering/implement-reflections-with-a-local-cubemap) @@ -203,11 +189,14 @@ Each scene can have a single environment light attached to it by defining the `e ### Environment Light Properties +The environment light declaration present in the root of the glTF object must contain one or more textures and images specification. +Textures are declared as an array of KHR_texture_ktx objects. Images are declared as an array of image objects. + `textures` | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| -| `textures` | texture [1-* ] | Array of textures. | :white_check_mark: Yes | +| `textures` | texture [1-* ] | Array of KHR_texture_ktx objects, referencing cubemap source images. | :white_check_mark: Yes | `images` @@ -223,9 +212,9 @@ Each scene can have a single environment light attached to it by defining the `e | `name` | String | Name of the light. | No | | `luminanceFactor` | number | Luminance factor for the environment, in candela per square meter. | No, Default: `1.0` | | `irradianceCoefficients` | number[9][3] | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | -| `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | :white_check_mark: Yes | -| `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | :white_check_mark: Yes | -| `cubemap` | integer | Reference to texture to be used as cubemap. The texture source may use some other extension to include additional texture format support. | :white_check_mark: Yes | +| `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | +| `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | +| `cubemap` | integer | Reference to texture source to be used as cubemap, source must contain valid cubemap. | :white_check_mark: Yes | ## KTX v2 Images From 904937bd8b1742b73564902405232f990ef94f07 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Mon, 18 Oct 2021 12:56:17 +0200 Subject: [PATCH 20/29] Add KHR_texture_ktx to textures array Textures referenced by the IBL extension shall use KHR_texture_ktx extension --- .../Khronos/KHR_lights_environment/README.md | 34 ++++++++++++------- 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 118f97a7cf..cd73f9c2f5 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -43,6 +43,7 @@ The number of specified mip-levels shall be log2 of the min value from texture w This is to ensure deterministic texture memory usage. Cubemap images are declared as an array of images in the extension that is placed in the glTF root. +These images are specified using KHR_texture_ktx extension to reference the source images for the textures used in the environment light extension. > **Implementation Note**: Implementations are free to ignore the pre-filtered mip-levels and generate the mip-levels for roughness values at runtime. @@ -53,9 +54,10 @@ The KHR_lights_environment extension defines an array of image-based lights at t Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for irradiance, luminance factor and bounding box for localized cubemap. The cubemap is defined as an integer reference to one of the texture sources declared in the this extension. -These textures may use an extension to specify the format, for instance using KHR_texture_basisu or KHR_texture_float_bc6h. +Textures may use an extension to specify the format, for instance KHR_texture_basisu. +If they do not use such an extension they must declare the textures with KHR_texture_ktx -When the extension is used, images shall use `image/ktx2` as mimeType. +When this extension is used, images shall use `image/ktx2` as mimeType. The texture type of the KTX v2 file shall be 'Cubemap' @@ -76,8 +78,12 @@ The following will load the environment light using KHR_texture_ktx. "KHR_lights_environment" : { "textures": [ { - "source": 0, - "layer": 0 + "source": 0, + "extensions": { + "KHR_texture_ktx": { + "source": 1 + "layer": 0 + } } ], "images": [ @@ -137,14 +143,17 @@ To avoid oversampling of flat surfaces (low roughness values) it is recommended ## Localized Cubemaps -The cubemaps shall use a technique called localized cubemaps, also called local cubemaps or box projected cubemaps. +The cubemaps may use a technique called localized cubemaps, also called local cubemaps or box projected cubemaps. This introduces a boundingbox (min / max) that makes it possible to calculate a local corrected reflection vector. -Meaning that models displayed within the cubemap environment does not have to rely on the centerpoint for light and reflection lookup. -Instead it is possible to place objects and the camera within this environment. -The cubemap boundingbox is declared as a pair of 3D coordinates for min / max. +Meaning that models displayed within the cubemap environment does not have to rely on the centerpoint for light and reflection lookup from an infinite distance. +Instead it is possible to place objects and the camera within this environment. The cubemap boundingbox is declared as a pair of 3D coordinates for min / max. + ### Implementation Note +If boundingbox min and max is not specified the distance to the cubemap shall be considered to be boundless. +This means that only directionality based on cubemap centerpoint shall be used. + Here is an example of a realtime implementation of reflections using local cubemap: [Implement reflections with a local cubemap](https://developer.arm.com/solutions/graphics-and-gaming/gaming-engine/unity/arm-guides-for-unity-developers/local-cubemap-rendering/implement-reflections-with-a-local-cubemap) @@ -190,19 +199,20 @@ Each scene can have a single environment light attached to it by defining the `e ### Environment Light Properties The environment light declaration present in the root of the glTF object must contain one or more textures and images specification. -Textures are declared as an array of KHR_texture_ktx objects. Images are declared as an array of image objects. +Textures must be declared using KHR_texture_ktx extension, or a texture extension that uses this. +Images are declared as an array of image objects. `textures` | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| -| `textures` | texture [1-* ] | Array of KHR_texture_ktx objects, referencing cubemap source images. | :white_check_mark: Yes | +| `textures` | texture [1-* ] | Array of textures, using KHR_texture_ktx or related extension, that reference the source image(s). Image source must contain valid cubemap. | :white_check_mark: Yes | `images` | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| -| `images` | image [1-* ] | Array of images that reference KTX V 2 file containing at least one cubemap. | :white_check_mark: Yes | +| `images` | image [1-* ] | Array of images that reference KTX V 2 file containing at least one cubemap. Mimetype must be 'image/ktx2'. | :white_check_mark: Yes | `light` @@ -210,7 +220,7 @@ Textures are declared as an array of KHR_texture_ktx objects. Images are declare | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| | `name` | String | Name of the light. | No | -| `luminanceFactor` | number | Luminance factor for the environment, in candela per square meter. | No, Default: `1.0` | +| `luminanceFactor` | number | Luminance factor for the environment, in candela per square meter. The luminanceFactor shall be applied both to specular cubemap and irradianceCOefficients, if thos are supplied. | No, Default: `1.0` | | `irradianceCoefficients` | number[9][3] | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | | `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | | `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | From a596a695cd54478b63bacfd5b96e09029e6bf4ea Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Thu, 21 Oct 2021 10:28:29 +0200 Subject: [PATCH 21/29] Clarification of light contribution Updated to be more precise on the specification of radiance cubemap and irradiance coefficients --- .../Khronos/KHR_lights_environment/README.md | 69 ++++++++++--------- 1 file changed, 37 insertions(+), 32 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index cd73f9c2f5..ce67b3e847 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -20,10 +20,15 @@ This extension depends on KHR_texture_ktx. This extension may use KHR_texture_basisu or KHR_texture_float_bc6h to support compressed texture formats. ## Overview -This extension provides the ability to define image-based lights in a glTF scene using KTX v2 images as defined by KHR_texture_ktx. -Image-based lights (environment map, light map) consist of a cubemap map that represents specular radiance for the scene, irradiance information and is the source for environment reflection. -This can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the IBL needs to be distributed. -It can also be used together with model data, for usecases where a model shall be displayed in a defined environment. +This extension provides the ability to define environment light contribution to a glTF using KTX v2 images, as defined by KHR_texture_ktx, and irradiance coefficients. +The extension provide two ways of supplying environmental light contribution. +Image-based by means of a cubemap and irradiance by means of spherical harmonics. + +The environment light cubemap can be seen as capturing the directed specular conntribution as well as the reflections. +The irradiance coefficients contain the non-directed light contribution from the scene. + +This extension can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the environmental light needs to be distributed. +It may also be used together with model data, for usecases where a model shall be displayed in a controlled environment. Many 3D tools and engines support image-based global illumination but the exact technique and data formats employed vary. Using this extension, tools can export and engines can import image-based lights and the result should be highly consistent. @@ -32,7 +37,12 @@ This extension specifies exactly one way to format and reference the environment First, it makes implementing support for this extension easier. Secondly, it ensures that rendering of the image-based lighting is consistent across runtimes. -A conforming implementation of this extension must be able to load the image-based environment data and render the PBR materials using this lighting. +A conforming implementation of this extension must be able to load the environment data and render the PBR materials using this lighting. + +Cubemap environment light images are declared as an array of images in the extension that is placed in the glTF root. +These images are specified using KHR_texture_ktx extension to reference the source images for the textures used in the environment light extension. + +### Pre Filtering Of Cubemaps The environment light is defined by a cubemap, this is to simplify realtime implementations as these are likely to use cubemap texture format. Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > 0, pre-filtering shall be done by the client. @@ -42,25 +52,19 @@ If a compressed texture format is used then pre-filtered mip-levels for roughnes The number of specified mip-levels shall be log2 of the min value from texture width and height, ie a full mipmap pyramid. This is to ensure deterministic texture memory usage. -Cubemap images are declared as an array of images in the extension that is placed in the glTF root. -These images are specified using KHR_texture_ktx extension to reference the source images for the textures used in the environment light extension. - - > **Implementation Note**: Implementations are free to ignore the pre-filtered mip-levels and generate the mip-levels for roughness values at runtime. ## Declaring An Environment Light -The KHR_lights_environment extension defines an array of image-based lights at the root of the glTF, each scene can reference one these lights. -Each environment light definition consists of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for irradiance, luminance factor and bounding box for localized cubemap. +The KHR_lights_environment extension defines an array of environment light cubemaps at the root of the glTF, each scene can reference one these lights. +Each environment light definition consists a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for irradiance, luminance factor and bounding box for localized cubemap. -The cubemap is defined as an integer reference to one of the texture sources declared in the this extension. -Textures may use an extension to specify the format, for instance KHR_texture_basisu. -If they do not use such an extension they must declare the textures with KHR_texture_ktx +The cubemap is defined as an integer reference to one of the texture sources declared in the this extension. +This texture source shall contain a cubemap at the layer defined by the texture. When this extension is used, images shall use `image/ktx2` as mimeType. The texture type of the KTX v2 file shall be 'Cubemap' - The following will load the environment light using KHR_texture_ktx. @@ -78,10 +82,9 @@ The following will load the environment light using KHR_texture_ktx. "KHR_lights_environment" : { "textures": [ { - "source": 0, "extensions": { "KHR_texture_ktx": { - "source": 1 + "source": 0 "layer": 0 } } @@ -109,14 +112,15 @@ The following will load the environment light using KHR_texture_ktx. ## Specular Radiance Cubemaps -The cubemap used for specular radiance is defined as a cubemap containing separate images for each cube face. -The mip levels shall evenly map to roughness values from 0 to 1 in the PBR material and should be generated with a principled multi-scatter GGX normal distribution. The data in the maps represents illuminance in candela per square meter. - -If mip-levels are included they are considered to be an approximation of the reflection at varying roughness levels. +The cubemap used for specular radiance is defined as a cubemap containing separate images for each cube face. +The data in the maps represents illuminance in candela per square meter. +Using this information it is possible to calculate the non-direct light contribution to the scene if irradiance coefficients are not included. Cube faces are defined in the KTX2 format specification, implementations of this extension must follow the KTX2 cubemap specification. -The boundingbox for the cubemaps are declared, this means that the locality of the camera within the cubemap can be calculated. +If specular radiance cubemap is supplied it shall be used as contribution to directed light incoming to the scene and as a source for environment reflections. + +If the boundingbox for the cubemaps are declared, this means that the locality of the camera within the cubemap can be calculated. Making it possible to calculate the localized reflection vector allowing for lighting and reflections that not always uses the centerpoint of the cubemap as a reference point. [See Localized cubemaps](#localized-cubemaps) @@ -124,20 +128,17 @@ Making it possible to calculate the localized reflection vector allowing for lig ### Implementation Note This section is non-normative -If mip-levels are not included, then the entire mip chain of images does not have be generated. -Generally it is enough to generate mip-level such that the lowest-resolution mip have sufficient size to represent the maximally-blurred radiance map corresponding to roughness=1. +The mip levels used for reflection shall evenly map to roughness values from 0 to 1 in the PBR material. +If needed these shall be generated by the implementation. +Exactly how this is done is up to the implementation, for a general realtime usecase it is usually enough to create the mip-levels using linear filtering. The texture references defines the largest dimension of mip 0 and should give the loading runtime the information it needs to generate the remainder of the mip chain and sample the appropriate mip level in the shader. - In a realtime renderer the cubemap textures may be used as a source for radiance, irradiance and reflection. A performant way of achieving this is to store the reflection at different roughness values in mip-levels, -where mip-level 0 is for a mirror like reflection (roughness = 0) and the last mip-level is for maximum roughness (roughness = 0). +where mip-level 0 is for a mirror like reflection (roughness = 0) and the last mip-level is for maximum roughness (roughness = 1). The lod level is calculated like this: lod = roughness * (numberOfMipLevels - 1) -It is recommended to map roughness = 1 to a mip-level size that is larger than 1 * 1 pixels to avoid blockiness, it is generally enough to stop at 16 * 16 pixels. -To avoid oversampling of flat surfaces (low roughness values) it is recommended to use a 3D cubemap when possible. - [Runtime filtering of mip-levels](https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-20-gpu-based-importance-sampling) [Creating prefiltered reflection map](https://docs.imgtec.com/Graphics_Techniques/PBR_with_IBL_for_PVR/topics/Assets/pbr_ibl__the_prefiltered_map.html) @@ -161,15 +162,19 @@ Here is an example of a realtime implementation of reflections using local cubem ## Irradiance Coefficients -This extension may use spherical harmonic coefficients to define irradiance used for diffuse lighting. +This extension may use spherical harmonic coefficients to define irradiance as a contribution for diffuse lighting. Coefficients are calculated for the first 3 SH bands (l=2) and take the form of a 9x3 array. -If irradiance coefficients are supplied the specular cubemaps shall not be used as a contribution to diffuse lighting calculations. + +If irradiance coefficients are supplied the specular radiance cubemaps shall not be used as a contribution to diffuse lighting calculations, instead the irradiance coefficients shall be used. + The occlusion parameter of a material will affect the contribution from the irradiance coefficients. +If occlusion is included in the material definition the diffuse contribution shall be calculated using the occlusion factor. + ### Implementation Note This section is non-normative -Implementations may calculate the irradiance from the specular radiance cubemap and use instead. +If irradiance coefficients are not defined, implementations may calculate irradiance from the specular radiance cubemap. One possible benefit with spherical harmonics is that it is generally enough evaluate the harmonics on a per vertex basis, instead of sampling a cubemap on a per fragment basis. [Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) From d2ffe33f01a8e5bb0e1df48a52021a1296af947a Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Thu, 21 Oct 2021 14:12:41 +0200 Subject: [PATCH 22/29] Remove luminance factor Since source images for the cubemap can be supplied using HDR (float) formats the luminance factor is removed. --- extensions/2.0/Khronos/KHR_lights_environment/README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index ce67b3e847..666400811e 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -99,7 +99,6 @@ The following will load the environment light using KHR_texture_ktx. "lights": [ { "name": "environment light 0", - "luminance": 1.0, "irradianceCoefficients": [...3 x 9 array of floats...], "boundingBoxMin": [-100, -100, -100], "boundingBoxMax": [100, 100, 100], @@ -225,11 +224,10 @@ Images are declared as an array of image objects. | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| | `name` | String | Name of the light. | No | -| `luminanceFactor` | number | Luminance factor for the environment, in candela per square meter. The luminanceFactor shall be applied both to specular cubemap and irradianceCOefficients, if thos are supplied. | No, Default: `1.0` | | `irradianceCoefficients` | number[9][3] | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | | `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | | `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | -| `cubemap` | integer | Reference to texture source to be used as cubemap, source must contain valid cubemap. | :white_check_mark: Yes | +| `cubemap` | integer | Reference to texture source to be used as specular radiance cubemap, source must contain valid cubemap. | :white_check_mark: Yes | ## KTX v2 Images From a8c21ec733daf7c18bcbdb59aa48cdb9edf4dddb Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Thu, 21 Oct 2021 14:14:06 +0200 Subject: [PATCH 23/29] Remove luminance from schema --- .../Khronos/KHR_lights_environment/schema/light.schema.json | 6 ------ 1 file changed, 6 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json index ec3beec94c..7ea48117d6 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json @@ -5,12 +5,6 @@ "description": "An image based environment light.", "allOf" : [ { "$ref" : "glTFChildOfRootProperty.schema.json" } ], "properties": { - "luminance": { - "type": "number", - "description": " Luminance factor for the environment, in candela per square meter. .", - "default": 1.0, - "minimum": 0.0 - }, "irradianceCoefficients": { "description": "Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array.", "type": "array", From 3232a99340253bfb7bdb63570acc2e762778a152 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Thu, 21 Oct 2021 14:21:40 +0200 Subject: [PATCH 24/29] Add contributor --- extensions/2.0/Khronos/KHR_lights_environment/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 666400811e..60df80748f 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -8,6 +8,7 @@ * Gary Hsu, Microsoft, * Mike Bond, Adobe, * Ben Houston, ThreeKit, +* Dominick D'Aniello, Mozilla ## Status From 00b37307c9f60d992bffc2956ba4ffc4701b73df Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Fri, 22 Oct 2021 15:54:33 +0200 Subject: [PATCH 25/29] Clarification of pre-filtered maps, add references --- .../Khronos/KHR_lights_environment/README.md | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 60df80748f..58016639b1 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -128,6 +128,10 @@ Making it possible to calculate the localized reflection vector allowing for lig ### Implementation Note This section is non-normative +One common way of sampling the reflected component of light contribution from the environment is using pre-filtered maps. +In short this is a process where the view-independent directional diffuse contribution is separated from the view-dependent ideal specular contribution. +This information is then stored as varying degrees of diffused vs specular light contribution in mip-levels of the cubemap and can then be used as a source for reflection from not only ideal specular surfaces. + The mip levels used for reflection shall evenly map to roughness values from 0 to 1 in the PBR material. If needed these shall be generated by the implementation. Exactly how this is done is up to the implementation, for a general realtime usecase it is usually enough to create the mip-levels using linear filtering. @@ -141,6 +145,7 @@ lod = roughness * (numberOfMipLevels - 1) [Runtime filtering of mip-levels](https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-20-gpu-based-importance-sampling) [Creating prefiltered reflection map](https://docs.imgtec.com/Graphics_Techniques/PBR_with_IBL_for_PVR/topics/Assets/pbr_ibl__the_prefiltered_map.html) +[A Global Illumination Solution for General Reflectance Distributions](https://www.graphics.cornell.edu/pubs/1991/SAWG91.pdf) ## Localized Cubemaps @@ -175,10 +180,7 @@ If occlusion is included in the material definition the diffuse contribution sha This section is non-normative If irradiance coefficients are not defined, implementations may calculate irradiance from the specular radiance cubemap. -One possible benefit with spherical harmonics is that it is generally enough evaluate the harmonics on a per vertex basis, instead of sampling a cubemap on a per fragment basis. - -[Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) -[An Efficient Representation for Irradiance Environment Maps](http://graphics.stanford.edu/papers/envmap/) +One possible benefit with spherical harmonics is that it is generally enough evaluate the harmonics on a per vertex basis, instead of sampling a cubemap on a per fragment basis. ### Using the environment light @@ -250,6 +252,12 @@ The texture type of the KTX v2 file shall be 'Cubemap' * `TODO: Add implementations` -## Reference +## References [Irradiance Environment Maps](https://graphics.stanford.edu/papers/ravir_thesis/chapter4.pdf) +[A Global Illumination Solution for General Reflectance Distributions](https://www.graphics.cornell.edu/pubs/1991/SAWG91.pdf) +[Realtime Image Based Lighting using Spherical Harmonics](https://metashapes.com/blog/realtime-image-based-lighting-using-spherical-harmonics/) +[An Efficient Representation for Irradiance Environment Maps](http://graphics.stanford.edu/papers/envmap/) +[Runtime filtering of mip-levels](https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-20-gpu-based-importance-sampling) +[Creating prefiltered reflection map](https://docs.imgtec.com/Graphics_Techniques/PBR_with_IBL_for_PVR/topics/Assets/pbr_ibl__the_prefiltered_map.html) + From 590c2ac5f534935112c85d79fac2f6f738338e76 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Wed, 8 Dec 2021 14:49:21 +0100 Subject: [PATCH 26/29] Remove dependency to KHR_texture_ktx This extension shall be standalone without the dependency to some other extension to define cubemap texture support. --- .../Khronos/KHR_lights_environment/README.md | 49 +++++++++---------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 58016639b1..10bc9eb4a1 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -17,16 +17,14 @@ Draft ## Dependencies Written against the glTF 2.0 spec. -This extension depends on KHR_texture_ktx. -This extension may use KHR_texture_basisu or KHR_texture_float_bc6h to support compressed texture formats. ## Overview -This extension provides the ability to define environment light contribution to a glTF using KTX v2 images, as defined by KHR_texture_ktx, and irradiance coefficients. +This extension provides the ability to define environment light contribution to a glTF using KTX v2 images and irradiance coefficients. The extension provide two ways of supplying environmental light contribution. Image-based by means of a cubemap and irradiance by means of spherical harmonics. The environment light cubemap can be seen as capturing the directed specular conntribution as well as the reflections. -The irradiance coefficients contain the non-directed light contribution from the scene. +The irradiance coefficients contain the non-directed light contribution from the scene, these may be supplied or can be calculated by the implementations. This extension can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the environmental light needs to be distributed. It may also be used together with model data, for usecases where a model shall be displayed in a controlled environment. @@ -40,8 +38,8 @@ Secondly, it ensures that rendering of the image-based lighting is consistent ac A conforming implementation of this extension must be able to load the environment data and render the PBR materials using this lighting. -Cubemap environment light images are declared as an array of images in the extension that is placed in the glTF root. -These images are specified using KHR_texture_ktx extension to reference the source images for the textures used in the environment light extension. +Cubemap environment light images are declared as an array of images in the extension, this extension is then placed in the glTF root. +These images shall be supplied using a KTX V2 file containing at least one cubemap. ### Pre Filtering Of Cubemaps @@ -49,11 +47,6 @@ The environment light is defined by a cubemap, this is to simplify realtime impl Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > 0, pre-filtering shall be done by the client. [See Specular radiance cubemaps](#specular-radiance-cubemaps) -If a compressed texture format is used then pre-filtered mip-levels for roughness values > 0 shall be specified. -The number of specified mip-levels shall be log2 of the min value from texture width and height, ie a full mipmap pyramid. -This is to ensure deterministic texture memory usage. - - > **Implementation Note**: Implementations are free to ignore the pre-filtered mip-levels and generate the mip-levels for roughness values at runtime. ## Declaring An Environment Light @@ -75,19 +68,15 @@ The following will load the environment light using KHR_texture_ktx. "version": "2.0" }, "extensionsUsed": [ - "KHR_texture_ktx", "KHR_lights_environment" ], "extensions": { "KHR_lights_environment" : { - "textures": [ + "cubemaps": [ { - "extensions": { - "KHR_texture_ktx": { - "source": 0 - "layer": 0 - } + "source": 0 + "layer": 0 } ], "images": [ @@ -125,7 +114,7 @@ Making it possible to calculate the localized reflection vector allowing for lig [See Localized cubemaps](#localized-cubemaps) -### Implementation Note +**Implementation Notes** This section is non-normative One common way of sampling the reflected component of light contribution from the environment is using pre-filtered maps. @@ -205,21 +194,22 @@ Each scene can have a single environment light attached to it by defining the `e ### Environment Light Properties -The environment light declaration present in the root of the glTF object must contain one or more textures and images specification. -Textures must be declared using KHR_texture_ktx extension, or a texture extension that uses this. -Images are declared as an array of image objects. +The environment light declaration present in the root of the glTF object must contain one or more cubemaps and images specification. +Cubemaps are declared as an array of image and layer indexes. +Images are declared as an array of image objects referencing KTX v2 image files. -`textures` +`cubemaps` | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| -| `textures` | texture [1-* ] | Array of textures, using KHR_texture_ktx or related extension, that reference the source image(s). Image source must contain valid cubemap. | :white_check_mark: Yes | +| `source` | integer | Index of KTX V2 file containing cubemap in the specified layer | :white_check_mark: Yes | +| `layer` | integer | Layer of the KTX V2 file containing the cubemap, if no value is supplied the default value of 0 is used | No | `images` | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| -| `images` | image [1-* ] | Array of images that reference KTX V 2 file containing at least one cubemap. Mimetype must be 'image/ktx2'. | :white_check_mark: Yes | +| `images` | Image [1-* ] | Array of images that reference KTX V 2 file containing at least one cubemap. Mimetype must be 'image/ktx2'. | :white_check_mark: Yes | `light` @@ -236,7 +226,14 @@ Images are declared as an array of image objects. ## KTX v2 Images The texture type of the KTX v2 file shall be 'Cubemap' - +The texture format must be one of the following: + +VK_FORMAT_R8G8B8_SRGB +VK_FORMAT_R8G8B8_UNORM +VK_FORMAT_E5B9G9R9_UFLOAT_PACK32 +VK_FORMAT_B10G11R11_UFLOAT_PACK32 +VK_FORMAT_R16G16B16_SFLOAT +VK_FORMAT_R16G16B16_UNORM ## glTF Schema Updates From bdbd58d7777d2881c9197d6e30f1b91ad9e44273 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Mon, 13 Dec 2021 12:39:58 +0100 Subject: [PATCH 27/29] Schema update Fixes to schema, add schema for cubemap --- .../Khronos/KHR_lights_environment/README.md | 5 +++-- .../schema/cubemap.schema.json | 21 +++++++++++++++++++ .../glTF.KHR_lights_environment.schema.json | 13 ++++++++++++ .../schema/light.schema.json | 2 +- 4 files changed, 38 insertions(+), 3 deletions(-) create mode 100644 extensions/2.0/Khronos/KHR_lights_environment/schema/cubemap.schema.json diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 10bc9eb4a1..5126f490a5 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -9,6 +9,7 @@ * Mike Bond, Adobe, * Ben Houston, ThreeKit, * Dominick D'Aniello, Mozilla +* Sebastian Vanderbeghe, ## Status @@ -59,7 +60,7 @@ This texture source shall contain a cubemap at the layer defined by the texture. When this extension is used, images shall use `image/ktx2` as mimeType. The texture type of the KTX v2 file shall be 'Cubemap' -The following will load the environment light using KHR_texture_ktx. +The following will declare one cubemap to be referenced by a scene. ```json @@ -220,7 +221,7 @@ Images are declared as an array of image objects referencing KTX v2 image files. | `irradianceCoefficients` | number[9][3] | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | | `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | | `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | -| `cubemap` | integer | Reference to texture source to be used as specular radiance cubemap, source must contain valid cubemap. | :white_check_mark: Yes | +| `cubemap` | integer | Reference to texture source to be used as specular radiance cubemap, this references one of the images declared by this extensions images array. | :white_check_mark: Yes | ## KTX v2 Images diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/cubemap.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/cubemap.schema.json new file mode 100644 index 0000000000..f0ce4c1989 --- /dev/null +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/cubemap.schema.json @@ -0,0 +1,21 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema", + "title": "light", + "type": "object", + "description": "Cubemap definition used for environment lights.", + "allOf" : [ { "$ref" : "glTFChildOfRootProperty.schema.json" } ], + "properties": { + "source": { + "description": "Image reference to one of the cubemap images declared by this extension.", + "type": "integer", + }, + "layer": { + "description": "Layer in the image that contains the cubemap, defaults to 0 if no value supplied.", + "type": "integer" + "extensions": { }, + "extras": { } + }, + "required": [ + "source", + ] +} diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json index 20fb95fc24..9c2bcc5fa1 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/glTF.KHR_lights_environment.schema.json @@ -6,12 +6,25 @@ "properties": { "lights": { "type": "array", + "description": "Array of lights to be used with this extension, referenced from a scene by the lights property.", "items": { "type": "object", "$ref": "light.schema.json" }, "minItems": 1 }, + "images": "array", + "description": "Array of image declarations, each image shall point to a KTX V2 file containing at least one cubemap.", + "items": { + "type": "object", + "$ref": "image.schema.json" + }, + "cubemaps": "array", + "description": "Array of cubemaps that are referenced by the environment light declaration in a scene.", + "items": { + "type": "object", + "$ref": "cubemap.schema.json" + } "extensions": { }, "extras": { } }, diff --git a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json index 7ea48117d6..4be7b72052 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json +++ b/extensions/2.0/Khronos/KHR_lights_environment/schema/light.schema.json @@ -21,7 +21,7 @@ }, "cubemap": { "type": "integer", - "description": "Texture reference to one of the declared cubemap textures.", + "description": "Texture reference to one of the declared cubemaps in this extension.", }, "boundingBoxMin": { "description": "Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters)", From 0b849a712ecb7d99b89b18c0e3bf0229fa5338f0 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Wed, 14 Jun 2023 15:14:16 +0200 Subject: [PATCH 28/29] Rename extension to KHR_environment_map What is specified here is not lights in the sense that are otherwise defined in glTF - for instance KHR_llights_punctual. One major difference is that the incoming light contribution from an environment lacks distance. This extension shall not be seen as a replacement for spot or directional lights - it shall be seen as an extension that may provide reflection lookup and some non-directed light contribution. But not as a replacement for 'real' light extensions - hence the decision to rename. --- .../Khronos/KHR_lights_environment/README.md | 72 +++++++++---------- 1 file changed, 36 insertions(+), 36 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 5126f490a5..2fc0d53a60 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -1,5 +1,5 @@ -# KHR_lights_environment +# KHR_environment_map ## Contributors @@ -20,39 +20,40 @@ Draft Written against the glTF 2.0 spec. ## Overview -This extension provides the ability to define environment light contribution to a glTF using KTX v2 images and irradiance coefficients. -The extension provide two ways of supplying environmental light contribution. -Image-based by means of a cubemap and irradiance by means of spherical harmonics. +This extension provides the ability to define one or moreenvironment maps to a glTF using KTX v2 cubemaps and/or irradiance coefficients. +The extension provide two ways of supplying the surrounding environment. +Texture-based by means of a cubemap and irradiance coefficients by means of spherical harmonics. -The environment light cubemap can be seen as capturing the directed specular conntribution as well as the reflections. -The irradiance coefficients contain the non-directed light contribution from the scene, these may be supplied or can be calculated by the implementations. +The texture cubemap can be seen as a way of representing the environment in which one or more glTF models are placed. +The irradiance coefficients contain the non-directed light contribution integrated from the scene, these may be supplied or can be calculated by the implementations. -This extension can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the environmental light needs to be distributed. +This extension can be used on it's own - ie a glTF asset with only environment map data - for a usecase where the environmental map needs to be distributed. It may also be used together with model data, for usecases where a model shall be displayed in a controlled environment. -Many 3D tools and engines support image-based global illumination but the exact technique and data formats employed vary. -Using this extension, tools can export and engines can import image-based lights and the result should be highly consistent. +Many 3D tools and engines support image-based 'global illumination' but the exact technique and data formats employed vary. +Using this extension, tools can export and engines can import environment maps and the result should be highly consistent. This extension specifies exactly one way to format and reference the environment map to be used, the goals of this are two-fold. First, it makes implementing support for this extension easier. -Secondly, it ensures that rendering of the image-based lighting is consistent across runtimes. +Secondly, it ensures that rendering of the environment map is consistent across runtimes. -A conforming implementation of this extension must be able to load the environment data and render the PBR materials using this lighting. +A conforming implementation of this extension must be able to load the environment data and render the PBR materials using this. -Cubemap environment light images are declared as an array of images in the extension, this extension is then placed in the glTF root. +Cubemap environment images are declared as an array of images in the extension, this extension is then placed in the glTF root. These images shall be supplied using a KTX V2 file containing at least one cubemap. ### Pre Filtering Of Cubemaps -The environment light is defined by a cubemap, this is to simplify realtime implementations as these are likely to use cubemap texture format. -Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > 0, pre-filtering shall be done by the client. +The environment is defined by a cubemap, this is to simplify realtime implementations as these are likely to use cubemap texture format. +Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > 0, pre-filtering shall be done by the client +- unless for compressed texture image formats that may include pre-filtered mip-maps. [See Specular radiance cubemaps](#specular-radiance-cubemaps) -## Declaring An Environment Light +## Declaring An Environment Map -The KHR_lights_environment extension defines an array of environment light cubemaps at the root of the glTF, each scene can reference one these lights. -Each environment light definition consists a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for irradiance, luminance factor and bounding box for localized cubemap. +The KHR_environment_map extension defines an array of environment cubemaps at the root of the glTF, each scene can reference one these cubemaps. +Each environment definition consists a single cubemap that describes the incoming radiance to the scene, the l=2 spherical harmonics coefficients for irradiance, luminance factor and bounding box for localized cubemap. The cubemap is defined as an integer reference to one of the texture sources declared in the this extension. This texture source shall contain a cubemap at the layer defined by the texture. @@ -69,11 +70,11 @@ The following will declare one cubemap to be referenced by a scene. "version": "2.0" }, "extensionsUsed": [ - "KHR_lights_environment" + "KHR_environment_map" ], "extensions": { - "KHR_lights_environment" : { + "KHR_environment_map" : { "cubemaps": [ { "source": 0 @@ -87,9 +88,9 @@ The following will declare one cubemap to be referenced by a scene. "mimeType": "image/ktx2" } ], - "lights": [ + "environment_maps": [ { - "name": "environment light 0", + "name": "environment map 0", "irradianceCoefficients": [...3 x 9 array of floats...], "boundingBoxMin": [-100, -100, -100], "boundingBoxMax": [100, 100, 100], @@ -100,7 +101,7 @@ The following will declare one cubemap to be referenced by a scene. } ``` -## Specular Radiance Cubemaps +## Radiance Cubemaps The cubemap used for specular radiance is defined as a cubemap containing separate images for each cube face. The data in the maps represents illuminance in candela per square meter. @@ -129,9 +130,8 @@ The texture references defines the largest dimension of mip 0 and should give th In a realtime renderer the cubemap textures may be used as a source for radiance, irradiance and reflection. A performant way of achieving this is to store the reflection at different roughness values in mip-levels, -where mip-level 0 is for a mirror like reflection (roughness = 0) and the last mip-level is for maximum roughness (roughness = 1). -The lod level is calculated like this: -lod = roughness * (numberOfMipLevels - 1) +where mip-level 0 is for a mirror like reflection (roughness = 0) and the last mip-level, where one texel lookup equals the normal distribution over the hemisphere (roughness = 1). + [Runtime filtering of mip-levels](https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-20-gpu-based-importance-sampling) [Creating prefiltered reflection map](https://docs.imgtec.com/Graphics_Techniques/PBR_with_IBL_for_PVR/topics/Assets/pbr_ibl__the_prefiltered_map.html) @@ -141,7 +141,7 @@ lod = roughness * (numberOfMipLevels - 1) The cubemaps may use a technique called localized cubemaps, also called local cubemaps or box projected cubemaps. This introduces a boundingbox (min / max) that makes it possible to calculate a local corrected reflection vector. -Meaning that models displayed within the cubemap environment does not have to rely on the centerpoint for light and reflection lookup from an infinite distance. +Meaning that models displayed within the cubemap environment does not have to rely on the centerpoint for reflection lookup from an infinite distance. Instead it is possible to place objects and the camera within this environment. The cubemap boundingbox is declared as a pair of 3D coordinates for min / max. @@ -157,7 +157,7 @@ Here is an example of a realtime implementation of reflections using local cubem ## Irradiance Coefficients -This extension may use spherical harmonic coefficients to define irradiance as a contribution for diffuse lighting. +This extension may use spherical harmonic coefficients to define irradiance as a contribution for diffuse (lambertian) lighting. Coefficients are calculated for the first 3 SH bands (l=2) and take the form of a 9x3 array. If irradiance coefficients are supplied the specular radiance cubemaps shall not be used as a contribution to diffuse lighting calculations, instead the irradiance coefficients shall be used. @@ -172,10 +172,10 @@ This section is non-normative If irradiance coefficients are not defined, implementations may calculate irradiance from the specular radiance cubemap. One possible benefit with spherical harmonics is that it is generally enough evaluate the harmonics on a per vertex basis, instead of sampling a cubemap on a per fragment basis. -### Using the environment light +### Using the environment map -The environment light is utilized by a scene. -Each scene can have a single environment light attached to it by defining the `extensions.KHR_lights_environment` property and, within that, an index into the `lights` array using the `light` property. +The environment map is utilized by a scene. +Each scene can have a single environment map attached to it by defining the `extensions.KHR_environment_map` property and, within that, an index into the `environment_maps` array using the `environment_map` property. ```json "scenes": [ @@ -184,8 +184,8 @@ Each scene can have a single environment light attached to it by defining the `e 0 ], "extensions": { - "KHR_lights_environment": { - "light": 0 + "KHR_environment_map": { + "environment_map": 0 } } } @@ -193,9 +193,9 @@ Each scene can have a single environment light attached to it by defining the `e ``` -### Environment Light Properties +### Environment Map Properties -The environment light declaration present in the root of the glTF object must contain one or more cubemaps and images specification. +The environment map declaration present in the root of the glTF object must contain one or more cubemaps and images specification. Cubemaps are declared as an array of image and layer indexes. Images are declared as an array of image objects referencing KTX v2 image files. @@ -213,11 +213,11 @@ Images are declared as an array of image objects referencing KTX v2 image files. | `images` | Image [1-* ] | Array of images that reference KTX V 2 file containing at least one cubemap. Mimetype must be 'image/ktx2'. | :white_check_mark: Yes | -`light` +`environment_map` | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| -| `name` | String | Name of the light. | No | +| `name` | String | Name of the environment map. | No | | `irradianceCoefficients` | number[9][3] | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | | `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | | `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | From c6241f9eec7b866225c7f35255d344a731fb5938 Mon Sep 17 00:00:00 2001 From: Richard Sahlin Date: Tue, 20 Jun 2023 13:23:51 +0200 Subject: [PATCH 29/29] Update README.md Moved images array to root of glTF. Added properties for cubemap intensity, irradiance factor and ior. --- .../Khronos/KHR_lights_environment/README.md | 28 +++++++++++-------- 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_lights_environment/README.md b/extensions/2.0/Khronos/KHR_lights_environment/README.md index 2fc0d53a60..a8d237db2d 100644 --- a/extensions/2.0/Khronos/KHR_lights_environment/README.md +++ b/extensions/2.0/Khronos/KHR_lights_environment/README.md @@ -52,7 +52,8 @@ Cubemaps shall be supplied without pre-filtered mip-maps for roughness values > ## Declaring An Environment Map -The KHR_environment_map extension defines an array of environment cubemaps at the root of the glTF, each scene can reference one these cubemaps. +The KHR_environment_map extension defines an array of environment cubemaps in the extensions root of the glTF, each scene can reference one these cubemaps. +Each environment may reference one cubemap, consisting of 6 textures. Cubemaps are declared in the root `images` array and reference a KTX V2 file. Each environment definition consists a single cubemap that describes the incoming radiance to the scene, the l=2 spherical harmonics coefficients for irradiance, luminance factor and bounding box for localized cubemap. The cubemap is defined as an integer reference to one of the texture sources declared in the this extension. @@ -81,13 +82,6 @@ The following will declare one cubemap to be referenced by a scene. "layer": 0 } ], - "images": [ - { - "name": "environment cubemap 0", - "uri": "cubemap0.ktx2", - "mimeType": "image/ktx2" - } - ], "environment_maps": [ { "name": "environment map 0", @@ -98,7 +92,15 @@ The following will declare one cubemap to be referenced by a scene. } ] } -} +}, +"images": [ + { + "name": "environment cubemap 0", + "uri": "cubemap0.ktx2", + "mimeType": "image/ktx2" + } +] + ``` ## Radiance Cubemaps @@ -203,9 +205,9 @@ Images are declared as an array of image objects referencing KTX v2 image files. | Property | Type | Description | Required | |:-----------------------|:-----------|:------------------------------------------| :--------------------------| -| `source` | integer | Index of KTX V2 file containing cubemap in the specified layer | :white_check_mark: Yes | -| `layer` | integer | Layer of the KTX V2 file containing the cubemap, if no value is supplied the default value of 0 is used | No | - +| `source` | integer | Index of KTX V2 file containing cubemap in the specified layer | :white_check_mark: Yes | +| `layer` | integer | Layer of the KTX V2 file containing the cubemap, if no value is supplied the default value of 0 is used | No | +| `intensity`| number | Intensity of cubemap values. The cubemap texel values shall be scaled by this value. Defaults to 1 if no value specified. | No | `images` | Property | Type | Description | Required | @@ -219,9 +221,11 @@ Images are declared as an array of image objects referencing KTX v2 image files. |:-----------------------|:-----------|:------------------------------------------| :--------------------------| | `name` | String | Name of the environment map. | No | | `irradianceCoefficients` | number[9][3] | Declares spherical harmonic coefficients for irradiance up to l=2. This is a 9x3 array. | No | +| `irradianceFactor` | Scale factor for irradiance coefficients. Each irradiance coefficient shall be scaled by this factor. Defaults to 1 of not specified | No | | `boundingBoxMin` | number[3] | Local boundingbox min. The minimum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | | `boundingBoxMax` | number[3] | Local boundingbox max. The maximum 3D point of the cubemap boundingbox. In world coordinates (meters) | No | | `cubemap` | integer | Reference to texture source to be used as specular radiance cubemap, this references one of the images declared by this extensions images array. | :white_check_mark: Yes | +| `ior` | number | Index of refraction of the media within the environment. If scene and environment is stand-alone the IOR is considered to be infinite. If glTF is referenced, for instance by glXF, the media shall be considered to be contained within the boundingbox. If no value is specified the IOR shall default to 1.0 for air. | No | ## KTX v2 Images