Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce new 'unsigned_long' numeric field type support #3585

Merged
merged 7 commits into from
May 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ nav_order: 43
has_children: false
parent: Object field types
grand_parent: Supported field types
redirect_from:
- /field-types/flat-object/
---

# Flat object field type
Expand Down
2 changes: 1 addition & 1 deletion _field-types/supported-field-types/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Field data type | Description
:--- | :---
[`alias`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/alias/) | An additional name for an existing field.
[`binary`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/binary/) | A binary value in Base64 encoding.
[Numeric]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/numeric/) | `byte`, `double`, `float`, `half_float`, `integer`, `long`, `scaled_float`, `short`.
[Numeric]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/numeric/) | `byte`, `double`, `float`, `half_float`, `integer`, `long`, `unsigned_long`, `scaled_float`, `short`.
[`boolean`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/boolean/) | A Boolean value.
[`date`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/date/) | A date value as a formatted string, a long value, or an integer.
[`ip`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/ip/) | An IP address in IPv4 or IPv6 format.
Expand Down
19 changes: 9 additions & 10 deletions _field-types/supported-field-types/numeric.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ layout: default
title: Numeric field types
parent: Supported field types
nav_order: 15
has_children: false
has_children: true
redirect_from:
- /opensearch/supported-field-types/numeric/
- /field-types/numeric/
Expand All @@ -15,15 +15,14 @@ The following table lists all numeric field types that OpenSearch supports.

Field data type | Description
:--- | :---
`byte` | A signed 8-bit integer. Minimum is -128. Maximum is 127.
`double` | A double-precision 64-bit IEEE 754 floating-point value. Minimum magnitude is 2<sup>-1074 </sup>. Maximum magnitude is (2 &minus; 2<sup>-52</sup>) &middot; 2<sup>1023</sup>. The number of significant bits is 53. The number of significant digits is 15.95.
`float` | A single-precision 32-bit IEEE 754 floating-point value. Minimum magnitude is 2<sup>-149 </sup>. Maximum magnitude is (2 &minus; 2<sup>-23</sup>) &middot; 2<sup>127</sup>. The number of significant bits is 24. The number of significant digits is 7.22.
`half_float` | A half-precision 16-bit IEEE 754 floating-point value. Minimum magnitude is 2<sup>-24 </sup>. Maximum magnitude is 65504. The number of significant bits is 11. The number of significant digits is 3.31.
`integer` | A signed 32-bit integer. Minimum is -2<sup>31</sup>. Maximum is 2<sup>31</sup> &minus; 1.
`long` | A signed 64-bit integer. Minimum is -2<sup>63</sup>. Maximum is 2<sup>63</sup> &minus; 1.
`short` | A signed 16-bit integer. Minimum is -2<sup>15</sup>. Maximum is 2<sup>15</sup> &minus; 1.

:--- | :---
`byte` | A signed 8-bit integer. Minimum is &minus;128. Maximum is 127.
`double` | A double-precision 64-bit IEEE 754 floating-point value. Minimum magnitude is 2<sup>&minus;1074 </sup>. Maximum magnitude is (2 &minus; 2<sup>&minus;52</sup>) &middot; 2<sup>1023</sup>. The number of significant bits is 53. The number of significant digits is 15.95.
`float` | A single-precision 32-bit IEEE 754 floating-point value. Minimum magnitude is 2<sup>&minus;149 </sup>. Maximum magnitude is (2 &minus; 2<sup>&minus;23</sup>) &middot; 2<sup>127</sup>. The number of significant bits is 24. The number of significant digits is 7.22.
`half_float` | A half-precision 16-bit IEEE 754 floating-point value. Minimum magnitude is 2<sup>&minus;24 </sup>. Maximum magnitude is 65504. The number of significant bits is 11. The number of significant digits is 3.31.
`integer` | A signed 32-bit integer. Minimum is &minus;2<sup>31</sup>. Maximum is 2<sup>31</sup> &minus; 1.
`long` | A signed 64-bit integer. Minimum is &minus;2<sup>63</sup>. Maximum is 2<sup>63</sup> &minus; 1.
[`unsigned_long`]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/unsigned-long/) | An unsigned 64-bit integer. Minimum is 0. Maximum is 2<sup>64</sup> &minus; 1.
`short` | A signed 16-bit integer. Minimum is &minus;2<sup>15</sup>. Maximum is 2<sup>15</sup> &minus; 1.
[`scaled_float`](#scaled-float-field-type) | A floating-point value that is multiplied by the double scale factor and stored as a long value.

Integer, long, float, and double field types have corresponding [range field types]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/range/).
Expand Down
164 changes: 164 additions & 0 deletions _field-types/supported-field-types/unsigned-long.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
---
layout: default
title: Unsigned long
parent: Numeric field types
grand_parent: Supported field types
nav_order: 15
has_children: false
---

# Unsigned long field type

The `unsigned_long` field type is a numeric field type that represents an unsigned 64-bit integer with a minimum value of 0 and a maximum value of 2<sup>64</sup> &minus; 1. In the following example, `counter` is mapped as an `unsigned_long` field:


```json
PUT testindex
{
"mappings" : {
"properties" : {
"counter" : {
"type" : "unsigned_long"
}
}
}
}
```
{% include copy-curl.html %}

## Indexing

To index a document with an `unsigned_long` value, use the following request:

```json
PUT testindex/_doc/1
{
"counter" : 10223372036854775807
}
```
{% include copy-curl.html %}

Alternatively, you can use the [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/) as follows:

```json
POST _bulk
{ "index": { "_index": "testindex", "_id": "1" } }
{ "counter": 10223372036854775807 }
```
{% include copy-curl.html %}

If a field of type `unsigned_long` has the `store` parameter set to `true` (that is, the field is a stored field), it will be stored and returned as a string. `unsigned_long` values do not support the decimal part, so, if supplied, the decimal part is truncated.
{: .note}

## Querying

`unsigned_long` fields support most of the queries that other numeric types support. For example, you can use a term query on `unsigned_long` fields:

```json
POST _search
{
"query": {
"term": {
"counter": {
"value": 10223372036854775807
}
}
}
}
```
{% include copy-curl.html %}

You can also use a range query:

```json
POST _search
{
"query": {
"range": {
"counter": {
"gte": 10223372036854775807
}
}
}
}
```
{% include copy-curl.html %}

## Sorting

You can use `sort` values with `unsigned_long` fields to order the search results, for example:

```json
POST _search
{
"sort" : [
{
"counter" : {
"order" : "asc"
}
}
],
"query": {
"range": {
"counter": {
"gte": 10223372036854775807
}
}
}
}
```
{% include copy-curl.html %}


An `unsigned_long` field cannot be used as an index sort field (in the `sort.field` index setting).
{: .warning}

## Aggregations

Like other numeric fields, `unsigned_long` fields support aggregations. For `terms` and `multi_terms` aggregations, `unsigned_long` values are used as is, but for other aggregation types, the values are converted to the `double` type (with possible loss of precision). The following is an example of the `terms` aggregation:

```json
POST _search
{
"query": {
"match_all": {}
},
"aggs": {
"counters": {
"terms": {
"field": "counter"
}
}
}
}
```
{% include copy-curl.html %}

## Scripting

In scripts, `unsigned_long` fields are returned as instances of the `BigInteger` class:

```json
POST _search
{
"query": {
"bool": {
"filter": {
"script": {
"script": "BigInteger amount = doc['counter'].value; return amount.compareTo(BigInteger.ZERO) > 0;"
}
}
}
}
}
```
{% include copy-curl.html %}


## Limitations

Note the following limitations of the `unsigned_long` field type:

- When aggregations are performed across different numeric types and one of the types is `unsigned_long`, the values are converted to the `double` type and `double` arithmetic is used, with high likelihood of precision loss.

- An `unsigned_long` field cannot be used as an index sort field (in the `sort.field` index setting). This limitation also applies when a search is performed on multiple indexes and the results are sorted by the field that has the `unsigned_long` type in at least one of the indexes but a different numeric type or types in others.