Prevent throttled indices to be searched through wildcards by default #34354
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today if a wildcard, date-math expression or alias expands/resolves to an index that is search-throttled we still search it. This is likely not the desired behavior since it can unexpectedly slow down searches significantly. This change adds a new indices option that allows `search`, `count` and `msearch` to ignore throttled indices by default. Users can force expansion to throttled indices by using `ignore_throttled=true` on the rest request to expand also to throttled indices. Relates to elastic#34352
s1monw
added
>enhancement
:Search/Search
|
Pinging @elastic/es-search-aggs |
javanna
changed the title
javanna
reviewed
Nov 5, 2018
server/src/main/java/org/elasticsearch/cluster/metadata/IndexNameExpressionResolver.java
Show resolved
Hide resolved
javanna
approved these changes
Nov 5, 2018
| @@ -233,6 +239,15 @@ | |||
| return concreteIndices.toArray(new Index[concreteIndices.size()]); | |||
| } | |||
|
|
|||
| private boolean addIndex(IndexMetaData metaData, Context context, Set<String> originalIndices) { | |||
s1monw
added a commit
that referenced
this pull request
Nov 6, 2018
…#34354) Today if a wildcard, date-math expression or alias expands/resolves to an index that is search-throttled we still search it. This is likely not the desired behavior since it can unexpectedly slow down searches significantly. This change adds a new indices option that allows `search`, `count` and `msearch` to ignore throttled indices by default. Users can force expansion to throttled indices by using `ignore_throttled=true` on the rest request to expand also to throttled indices. Relates to #34352
matarrese
added a commit
to matarrese/elasticsearch
that referenced
this pull request
Nov 6, 2018
…-agg * master: (528 commits) Register Azure max_retries setting (elastic#35286) add version 6.4.4 [Docs] Add painless context details for bucket_script (elastic#35142) Upgrade jline to 3.8.2 (elastic#35288) SQL: new SQL CLI logo (elastic#35261) Logger: Merge ESLoggerFactory into Loggers (elastic#35146) Docs: Add section about range query for range type (elastic#35222) [ILM] change remove-policy-from-index http method from DELETE to POST (elastic#35268) [CCR] Forgot missing return statement, SQL: Fix null handling for AND and OR in SELECT (elastic#35277) [TEST] Mute ChangePolicyForIndexIT#testChangePolicyForIndex Serialize ignore_throttled also to 6.6 after backport Check for java 11 in buildSrc (elastic#35260) [TEST] increase await timeout in RemoteClusterConnectionTests Add missing up-to-date configuration (elastic#35255) Adapt Lucene BWC version SQL: Introduce Coalesce function (elastic#35253) Upgrade to lucene-8.0.0-snapshot-31d7dfe6b1 (elastic#35224) Fix failing ICU tests (elastic#35207) Prevent throttled indices to be searched through wildcards by default (elastic#34354) ...
s1monw
added a commit
to s1monw/elasticsearch
that referenced
this pull request
Nov 7, 2018
Today we only apply `ingore_throttled` to expansions from wildcards, date math expressions and aliases. Yet, this is tricky since we might have resolved certain expressions in pre-filter steps like security. It's more consistent to apply this logic to all expressions including concrete indices. Relates to elastic#34354
s1monw
added a commit
that referenced
this pull request
Nov 7, 2018
Today we only apply `ingore_throttled` to expansions from wildcards, date math expressions and aliases. Yet, this is tricky since we might have resolved certain expressions in pre-filter steps like security. It's more consistent to apply this logic to all expressions including concrete indices. Relates to #34354
s1monw
added a commit
that referenced
this pull request
Nov 7, 2018
Today we only apply `ingore_throttled` to expansions from wildcards, date math expressions and aliases. Yet, this is tricky since we might have resolved certain expressions in pre-filter steps like security. It's more consistent to apply this logic to all expressions including concrete indices. Relates to #34354
pgomulka
pushed a commit
to pgomulka/elasticsearch
that referenced
this pull request
Nov 13, 2018
Today we only apply `ingore_throttled` to expansions from wildcards, date math expressions and aliases. Yet, this is tricky since we might have resolved certain expressions in pre-filter steps like security. It's more consistent to apply this logic to all expressions including concrete indices. Relates to elastic#34354
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today if a wildcard, date-math expression or alias expands/resolves
to an index that is search-throttled we still search it. This is likely
not the desired behavior since it can unexpectedly slow down searches
significantly.
This change adds a new indices option that allows
search,countand
msearchto ignore throttled indices by default. Users canforce expansion to throttled indices by using
ignore_throttled=trueon the rest request to expand also to throttled indices.
Relates to #34352
Relates to #33732