Batch Table Binary and other updates

README.md

@@ -172,7 +172,7 @@ An optional `transform` property (not shown above) defines a 4x4 affine transfor
 
 ### Coordinate System and Units
 
-Like glTF, 3D Tiles use a right-handed Cartesian coordinate system, that is, the cross product of x and y yields z. Also like glTF, 3D Tiles define the y axis as up for local Cartesian coordinate systems (see the [Tile transform](#tile-transform) section).  A tileset's global coordinate system will often be [WGS84 coordinates](http://earth-info.nga.mil/GandG/publications/tr8350.2/wgs84fin.pdf), but it doesn't have to be, e.g., a power plant may be defined fully in its local coordinate system for using with a modeling tool without a geospatial context.
+3D Tiles use a right-handed Cartesian coordinate system, that is, the cross product of x and y yields z. 3D Tiles define the z axis as up for local Cartesian coordinate systems (see the [Tile transform](#tile-transform) section).  A tileset's global coordinate system will often be [WGS84 coordinates](http://earth-info.nga.mil/GandG/publications/tr8350.2/wgs84fin.pdf), but it doesn't have to be, e.g., a power plant may be defined fully in its local coordinate system for using with a modeling tool without a geospatial context.
 
 The units for all linear distances are meters.
 
@@ -359,7 +359,7 @@ The top-level object in tileset.json has four properties: `asset`, `properties`,
 
 `asset` is an object containing properties with metadata about the entire tileset.  Its `version` property is a string that defines the 3D Tiles version.  The version defines the JSON schema for tileset.json and the base set of tile formats.  The `tilesetVersion` property is an optional string that defines an application-specific version of a tileset, e.g., for when an existing tileset is updated.
 
-`properties` is an object containing objects for each per-feature property in the tileset.  This tileset.json snippet is for 3D buildings, so each tile has building models, and each building model has a `Height` property (see the _Batch Table_ in the [Batched 3D Model](TileFormats/Batched3DModel/README.md) tile format).  The name of each object in `properties` matches the name of a per-feature property, and defines its `minimum` and `maximum` numeric values, which are useful, for example, for creating color ramps for styling.
+`properties` is an object containing objects for each per-feature property in the tileset.  This tileset.json snippet is for 3D buildings, so each tile has building models, and each building model has a `Height` property (see (Batch Table)[TileFormats/BatchTable/README.md]).  The name of each object in `properties` matches the name of a per-feature property, and defines its `minimum` and `maximum` numeric values, which are useful, for example, for creating color ramps for styling.
 
 `geometricError` is a nonnegative number that defines the error, in meters, when the tileset is not rendered.
 

TileFormats/BatchTable/README.md

@@ -0,0 +1,163 @@
+# Batch Table
+
+## Contributors
+
+* Sean Lilley, [@lilleyse](https://github.com/lilleyse)
+* Rob Taglang, [@lasalvavida](https://github.com/lasalvavida)
+* Tom Fili, [@CesiumFili](https://twitter.com/CesiumFili)
+* Patrick Cozzi, [@pjcozzi](https://twitter.com/pjcozzi)
+
+## Overview
+
+A _Batch Table_ contains per-feature application-specific metadata in a tile. These properties may be queried at runtime for declarative styling and application-specific use cases such as populating a UI or issuing a REST API request.  Some example Batch Table properties are building heights, cartographic coordinates, and database primary keys.
+
+A Batch Table is used by the following tile formats:
+* [Batched 3D Model](../Batched3DModel/README.md) (b3dm)
+* [Instanced 3D Model](../Instanced3DModel/README.md) (i3dm)
+* [Point Cloud](../PointCloud/README.md) (pnts)
+* [Vector](../VectorData/README.md) (vctr)
+
+## Layout
+
+A Batch Table is composed of two parts: a JSON header and an optional binary body. The JSON describes the properties, whose values can either be defined directly in the JSON as an array, or refer to sections in the binary body.  It is more efficient to store long numeric arrays in the binary body.
+
+**Figure 1**: Batch Table layout
+
+![batch table layout](figures/batch-table-layout.png)
+
+When a tile format includes a Batch Table, the Batch Table immediately follows the tile's Feature Table if it exists.  Otherwise, the Batch Table immediately follows the tile's header.
+The header will also contain `batchTableJSONByteLength` and `batchTableBinaryByteLength` `uint32` fields, which can be used to extract each respective part of the Batch Table.
+
+Code for reading the Batch Table can be found in [Cesium3DTileBatchTable.js](https://github.com/AnalyticalGraphicsInc/cesium/blob/3d-tiles/Source/Scene/Cesium3DTileBatchTable.js) in the Cesium implementation of 3D Tiles.
+
+## JSON Header
+
+Batch Table values can be represented in the JSON header in two different ways.
+
+1. An array of values. (e.g. `"name" : ['name1', 'name2', 'name3']` or `"height" : [10.0, 20.0, 15.0]`)
+    * Array elements can be any valid JSON data type, including objects and arrays.  Elements may be `null`.
+    * The length of each array is equal to `batchLength` which is specified in each tile format.  This is the number of features in the tile.  For example, `batchLength` may be the number of models in a b3dm tile, the number of instances in a i3dm tile, or the number of points (or number of objects) in a pnts tile.
+2. A reference to data in the binary body, denoted by an object with `byteOffset`, `componentType`, and `type` properties property. (e.g. `"height" : { "byteOffset" : 24, "componentType" : "FLOAT", "type" : "SCALAR"}`).
+    * `byteOffset` is a zero-based offset relative to the start of the binary body.
+    * `componentType` is the datatype of components in the attribute. Allowed values are `"BYTE"`, `"UNSIGNED_BYTE"`, `"SHORT"`, `"UNSIGNED_SHORT"`, `"INT"`, `"UNSIGNED_INT"`, `"FLOAT"`, and `"DOUBLE"`.
+    * `type` specifies if the property is a scalar, vector, or matrix. Allowed values are `"SCALAR"`, `"VEC2"`, `"VEC3"`, `"VEC4"`, `"MAT2"`, `"MAT3"`, and `"MAT4"`. Matrices are stored in the binary body in column-major order.
+
+The Batch Table JSON is a `UTF-8` string containing JSON. It can be extracted from the arraybuffer using the `TextDecoder` JavaScript API and transformed to a JavaScript object with `JSON.parse`.
+
+A `batchId` is used to access elements in each array and extract the corresponding properties. For example, the following Batch Table has properties for a batch of two features:
+```json
+{
+    "id" : ["unique id", "another unique id"],
+    "displayName" : ["Building name", "Another building name"],
+    "yearBuilt" : [1999, 2015],
+    "address" : [{"street" : "Main Street", "houseNumber" : "1"}, {"street" : "Main Street", "houseNumber" : "2"}]
+}
+```
+
+The properties for the feature with `batchId = 0` are
+```javascript
+id[0] = 'unique id';
+displayName[0] = 'Building name';
+yearBuilt[0] = 1999;
+address[0] = {street : 'Main Street', houseNumber : '1'};
+```
+
+The properties for `batchId = 1` are
+```javascript
+id[1] = 'another unique id';
+displayName[1] = 'Another building name';
+yearBuilt[1] = 2015;
+address[1] = {street : 'Main Street', houseNumber : '2'};
+```
+
+JSON Schema Batch Table definitions can be found in [batchTable.schema.json](../../schema/batchTable.schema.json).
+
+## Binary Body
+
+When the JSON header includes a reference to the binary section, the provided `byteOffset` is used to index into the data. 
+
+**Figure 2**: Indexing into the Batch Table binary body
+
+![batch table binary index](figures/batch-table-binary-index.png)
+
+Values can be retrieved using the number of features, `batchLength`, the desired batch id, `batchId`, and the `componentType` and `type` defined in the JSON header.
+
+The following tables can be used to compute the byte size of a property.
+
+| `componentType` | Size in bytes |
+| --- | --- |
+| `"BYTE"` | 1 |
+| `"UNSIGNED_BYTE"` | 1 |
+| `"SHORT"` | 2 |
+| `"UNSIGNED_SHORT"` | 2 |
+| `"INT"` | 4 |
+| `"UNSIGNED_INT"` | 4 |
+| `"FLOAT"` | 4 |
+| `"DOUBLE"` | 8 |
+
+| `type` | Number of components |
+| --- | --- |
+| `"SCALAR"` | 1 |
+| `"VEC2"` | 2 |
+| `"VEC3"` | 3 |
+| `"VEC4"` | 4 |
+| `"MAT2"` | 4 |
+| `"MAT3"` | 9 |
+| `"MAT4"` | 16 |
+
+For example, given the following Batch Table JSON with `batchLength` of 10
+
+```json
+{
+    "height" : {
+        "byteOffset" : 0,
+        "componentType" : "FLOAT",
+        "type" : "SCALAR"
+    },
+    "cartographic" : {
+        "byteOffset" : 40,
+        "componentType" : "DOUBLE",
+        "type" : "VEC3"
+    }
+}
+```
+
+To get the `"height"` values:
+
+```javascript
+var height = batchTableJSON.height;
+var byteOffset = height.byteOffset;
+var componentType = height.componentType;
+var type = height.type;
+
+var heightArrayByteLength = batchLength * sizeInBytes(componentType) * numberOfComponents(type); // 10 * 4 * 1
+var heightArray = new Float32Array(batchTableBinary.buffer, byteOffset, heightArrayByteLength);
+var heightOfFeature = heightArray[batchId];
+```
+
+To get the `"cartographic"` values:
+
+```javascript
+var cartographic = batchTableJSON.cartographic;
+var byteOffset = cartographic.byteOffset;
+var componentType = cartographic.componentType;
+var type = cartographic.type;
+var componentSizeInBytes = sizeInBytes(componentType)
+var numberOfComponents = numberOfComponents(type);
+
+var cartographicArrayByteLength = batchLength * componentSizeInBytes * numberOfComponents // 10 * 8 * 3
+var cartographicArray = new Float64Array(batchTableBinary.buffer, byteOffset, cartographicArrayByteLength);
+var cartographicOfFeature = positionArray.subarray(batchId * numberOfComponents, batchId * numberOfComponents + numberOfComponents); // Using subarray creates a view into the array, and not a new array.
+```
+
+## Implementation Notes
+
+In JavaScript, a `TypedArray` cannot be created on data unless it is byte-aligned to the data type.
+For example, a `Float32Array` must be stored in memory such that its data begins on a byte multiple of four since each `float` contains four bytes.
+
+The string generated from the JSON header should be padded with space characters in order to ensure that the binary body is byte-aligned.
+The binary body should also be padded if necessary when there is data following the Batch Table.
+
+## Acknowledgments
+
+* Jannes Bolling, [@jbo023](https://github.com/jbo023)

TileFormats/Batched3DModel/README.md

@@ -23,55 +23,28 @@ A tile is composed of two sections: a header immediately followed by a body.
 
 ## Header
 
-The 20-byte header contains the following fields:
+The 24-byte header contains the following fields:
 
 |Field name|Data type|Description|
 |----------|---------|-----------|
 | `magic` | 4-byte ANSI string | `"b3dm"`.  This can be used to identify the arraybuffer as a Batched 3D Model tile. |
 | `version` | `uint32` | The version of the Batched 3D Model format. It is currently `1`. |
 | `byteLength` | `uint32` | The length of the entire tile, including the header, in bytes. |
+| `batchTableJSONByteLength` | `uint32` | The length of the batch table JSON section in bytes. Zero indicates there is no batch table. |
+| `batchTableBinaryByteLength` | `uint32` | The length of the batch table binary section in bytes. If `batchTableJSONByteLength` is zero, this will also be zero. |
 | `batchLength` | `unit32` | The number of models, also called features, in the batch. |
-| `batchTableByteLength` | `uint32` | The length of the batch table in bytes. Zero indicates there is not a batch table. |
-
-_TODO: Link to Cesium code for reading header_
 
 The body section immediately follows the header section, and is composed of two fields: `Batch Table` and `Binary glTF`.
 
-## Batch Table
-
-In the Binary glTF section, each vertex has an unsigned short `batchId` attribute in the range `[0, number of models in the batch - 1]`.  The `batchId` indicates the model to which the vertex belongs.  This allows models to be batched together and still be identifiable.
-
-The batch table maps each `batchId` to per-model properties.  If present, the batch table immediately follows the header and is `batchTableByteLength` bytes long.
-
-The batch table is a `UTF-8` string containing JSON.  It immediately follows the header.  It can be extracted from the arraybuffer using the `TextDecoder` JavaScript API and transformed to a JavaScript object with `JSON.parse`.
-
-Each property in the object is an array with its length equal to `header.batchLength`.  Array elements can be any valid JSON data type, including objects and arrays.  Elements may be `null`.
+Code for reading the header can be found in
+[Batched3DModelTileContent](https://github.com/AnalyticalGraphicsInc/cesium/blob/3d-tiles/Source/Scene/Batched3DModel3DTileContent.js)
+in the Cesium implementation of 3D Tiles.
 
-A vertex's `batchId` is used to access elements in each array and extract the corresponding properties.  For example, the following batch table has properties for a batch of two models.
-```json
-{
-    "id" : ["unique id", "another unique id"],
-    "displayName" : ["Building name", "Another building name"],
-    "yearBuilt" : [1999, 2015],
-    "address" : [{"street" : "Main Street", "houseNumber" : "1"}, {"street" : "Main Street", "houseNumber" : "2"}]
-}
-```
+## Batch Table
 
-The properties for the model with `batchId = 0` are
-```javascript
-id[0] = 'unique id';
-displayName[0] = 'Building name';
-yearBuilt[0] = 1999;
-address[0] = {street : 'Main Street', houseNumber : '1'};
-```
+The _Batch Table_ contains per-model application-specific metadata, indexable by `batchId`, that can be used for declarative styling and application-specific use cases such as populating a UI or issuing a REST API request.  In the Binary glTF section, each vertex has an unsigned short `batchId` attribute in the range `[0, number of models in the batch - 1]`.  The `batchId` indicates the model to which the vertex belongs.  This allows models to be batched together and still be identifiable.
 
-The properties for `batchId = 1` are
-```javascript
-id[1] = 'another unique id';
-displayName[1] = 'Another building name';
-yearBuilt[1] = 2015;
-address[1] = {street : 'Main Street', houseNumber : '2'};
-```
+See the [Batch Table](../BatchTable/README.md) reference for more information.
 
 ## Binary glTF
 
@@ -114,7 +87,3 @@ Although not strictly required, clients may find the glTF [CESIUM_RTC](https://g
 _TODO, [#60](https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/60)_
 
 `application/octet-stream`
-
-## Acknowledgments
-
-* Jannes Bolling, [@jbo023](https://github.com/jbo023)

TileFormats/Composite/README.md

@@ -2,7 +2,7 @@
 
 ## Contributors
 
-* Sean Lilley, [@lilleyse](https://twitter.com/lilleyse)
+* Sean Lilley, [@lilleyse](https://github.com/lilleyse)
 * Patrick Cozzi, [@pjcozzi](https://twitter.com/pjcozzi)
 
 ## Overview

TileFormats/FeatureTable/README.md

@@ -2,19 +2,19 @@
 
 ## Contributors
 
-* Sean Lilley, [@lilleyse](https://twitter.com/lilleyse)
+* Sean Lilley, [@lilleyse](https://github.com/lilleyse)
 * Rob Taglang, [@lasalvavida](https://github.com/lasalvavida)
 * Dan Bagnell, [@bagnell](https://github.com/bagnell)
 * Patrick Cozzi, [@pjcozzi](https://twitter.com/pjcozzi)
 
 ## Overview
 
-A _Feature Table_ describes position and appearance properties for each feature in a tile.  The _Batch Table_ (TODO: link), on the other hand, contains per-feature application-specific metadata not necessarily used for rendering.
+A _Feature Table_ describes position and appearance properties for each feature in a tile.  The [Batch Table](../BatchTable/README.md), on the other hand, contains per-feature application-specific metadata not necessarily used for rendering.
 
 A Feature Table is used by the following tile formats:
-* [Instanced 3D Model](../Instanced3DModel) (i3dm) - each model instance is a feature.
-* [Point Cloud](../PointCloud) (pnts) - each point is a feature.
-* [Vector](../VectorData) (vctr) - each point/polyline/polygon is a feature.
+* [Instanced 3D Model](../Instanced3DModel/README.md) (i3dm) - each model instance is a feature.
+* [Point Cloud](../PointCloud/README.md) (pnts) - each point is a feature.
+* [Vector](../VectorData/README.md) (vctr) - each point/polyline/polygon is a feature.
 
 Per-feature properties are defined using tile-format-specific semantics defined in each tile format's specification.  For example, in _Instanced 3D Model_, `SCALE_NON_UNIFORM` defines the non-uniform scale applied to each 3D model instance.
 
@@ -28,7 +28,7 @@ A Feature Table is composed of two parts: a JSON header and an optional binary b
 
 When a tile format includes a Feature Table, the Feature Table immediately follows the tile's header.  The header will also contain `featureTableJSONByteLength` and `featureTableBinaryByteLength` `uint32` fields, which can be used to extract each respective part of the Feature Table.
 
-Code for reading the Feature Table can be found in [Cesium3DTileFeatureTableResources.js](https://github.com/AnalyticalGraphicsInc/cesium/blob/3d-tiles/Source/Scene/Cesium3DTileFeatureTableResources.js) in the Cesium implementation of 3D Tiles.
+Code for reading the Feature Table can be found in [Cesium3DTileFeatureTable.js](https://github.com/AnalyticalGraphicsInc/cesium/blob/3d-tiles/Source/Scene/Cesium3DTileFeatureTable.js) in the Cesium implementation of 3D Tiles.
 
 ## JSON Header
 
@@ -40,7 +40,8 @@ Feature Table values can be represented in the JSON header in three different wa
    * This is used for per-feature semantics like `"POSITION"` in Instanced 3D Model.  Above, each `POSITION` refers to a `float32[3]` data type so there are three features: `Feature 0's position`=`(1.0, 0.0, 0.0)`, `Feature 1's position`=`(0.0, 1.0, 0.0)`, `Feature 2's position`=`(0.0, 0.0, 1.0)`.
 3. A reference to data in the binary body, denoted by an object with a `byteOffset` property. (e.g. `"SCALE" : { "byteOffset" : 24}`).
    * `byteOffset` is a zero-based offset relative to the start of the binary body.
-   * The semantic defines the allowed data type, e.g., when `"POSITION"` in Instanced Model refers to the binary body, the component type is `float32` and the number of components is `3`.
+   * The semantic defines the allowed data type, e.g., when `"POSITION"` in Instanced Model refers to the binary body, the component type is `FLOAT` and the number of components is `3`.
+   * Some semantics allow for overriding the implicit `componentType`. These cases are specified in each tile format. (e.g. `"BATCH_ID" : { "byteOffset" : 24, "componentType" : "UNSIGNED_BYTE"}`).
 The only valid properties in the JSON header are the defined semantics by the tile format.  Application-specific data should be stored in the Batch Table.
 
 JSON Schema Feature Table definitions can be found in [featureTable.schema.json](../../schema/featureTable.schema.json).
@@ -58,7 +59,7 @@ Values can be retrieved using the number of features, `featuresLength`, the desi
 For example, using the `POSITION` semantic, which has a `float32[3]` data type:
 
 ```javascript
-var byteOffset = featureTableJSON.POSTION.byteOffset;
+var byteOffset = featureTableJSON.POSITION.byteOffset;
 
 var positionArray = new Float32Array(featureTableBinary.buffer, byteOffset, featuresLength * 3); // There are three components for each POSITION feature.
 var position = positionArray.subarray(featureId * 3, featureId * 3 + 3); // Using subarray creates a view into the array, and not a new array.