Skip to content

Commit

Permalink
Clarify the settings around limiting nested mappings. (#42686)
Browse files Browse the repository at this point in the history
* Previously, we mentioned multiple times that each nested object was indexed as its own document. This is repetitive, and is also a bit confusing in the context of `index.mapping.nested_fields.limit`, as that applies to the number of distinct `nested` types in the mappings, not the number of nested objects. We now just describe the issue once at the beginning of the section, to illustrate why `nested` types can be expensive.
* Reference the ongoing example to clarify the meaning of the two settings.

Addresses #28363.
  • Loading branch information
jtibshirani committed May 30, 2019
1 parent c181635 commit 1bb505c
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 22 deletions.
10 changes: 3 additions & 7 deletions docs/reference/mapping.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -87,15 +87,11 @@ causing a mapping explosion:
`2`, etc. The default is `20`.

`index.mapping.nested_fields.limit`::
The maximum number of `nested` fields in an index, defaults to `50`.
Indexing 1 document with 100 nested fields actually indexes 101 documents
as each nested document is indexed as a separate hidden document.
The maximum number of distinct `nested` mappings in an index, defaults to `50`.

`index.mapping.nested_objects.limit`::
The maximum number of `nested` json objects within a single document across
all nested fields, defaults to 10000. Indexing one document with an array of
100 objects within a nested field, will actually create 101 documents, as
each nested object will be indexed as a separate hidden document.
The maximum number of `nested` JSON objects within a single document across
all nested types, defaults to 10000.

`index.mapping.field_name_length.limit`::
Setting for the maximum length of a field name. The default value is
Expand Down
42 changes: 27 additions & 15 deletions docs/reference/mapping/types/nested.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -193,20 +193,32 @@ phase. Instead, highlighting needs to be performed via
=============================================

[[limit-number-nested-fields]]
==== Limiting the number of `nested` fields

Indexing a document with 100 nested fields actually indexes 101 documents as each nested
document is indexed as a separate document. To safeguard against ill-defined mappings
the number of nested fields that can be defined per index has been limited to 50. See
<<mapping-limit-settings>>.

[[limit-nested-json-objects-number]]
==== Limiting the number of `nested` json objects
Indexing a document with an array of 100 objects within a nested field, will actually
create 101 documents, as each nested object will be indexed as a separate document.
To prevent out of memory errors when a single document contains too many nested json
objects, the number of nested json objects that a single document may contain across all fields
has been limited to 10000. See <<mapping-limit-settings>>.
[float]
=== Limits on `nested` mappings and objects

As described earlier, each nested object is indexed as a separate document under the hood.
Continuing with the example above, if we indexed a single document containing 100 `user` objects,
then 101 Lucene documents would be created -- one for the parent document, and one for each
nested object. Because of the expense associated with `nested` mappings, Elasticsearch puts
settings in place to guard against performance problems:

`index.mapping.nested_fields.limit`::

The `nested` type should only be used in special cases, when arrays of objects need to be
queried independently of each other. To safeguard against poorly designed mappings, this setting
limits the number of unique `nested` types per index. In our example, the `user` mapping would
count as only 1 towards this limit. Defaults to 50.

`index.mapping.nested_objects.limit`::

This setting limits the number of nested objects that a single document may contain across all
`nested` types, in order to prevent out of memory errors when a document contains too many nested
objects. To illustrate how the setting works, say we added another `nested` type called `comments`
to our example mapping above. Then for each document, the combined number of `user` and `comment`
objects it contains must be below the limit. Defaults to 10000.

Additional background on these settings, including information on their default values, can be found
in <<mapping-limit-settings>>.



0 comments on commit 1bb505c

Please sign in to comment.