Backport/backport 6834 to 2.x #6893
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
) Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>
…ch-project#5928) Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]>
Signed-off-by: Anshu Agarwal <[email protected]>
* Add artifacts from benchmarks to .gitignore Signed-off-by: Peter Nied <[email protected]> * Update the section title Signed-off-by: Peter Nied <[email protected]> Signed-off-by: Peter Nied <[email protected]>
) Previously this test required the feature flag to be set as a system property, otherwise the test did nothing. This change programmatically sets the feature flag to allow the test to run during all builds. I ran the following command overnight (probably at least a couple thousand iterations) on a dev machine and saw no failures, so I have confidence this test is not flaky: ``` run-one-until-failure ./gradlew ':server:internalClusterTest' --tests "org.opensearch.snapshots.SearchableSnapshotIT" ``` Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]>
…rmark (opensearch-project#5852) * Adding index create block when all nodes have breached high disk watermark Signed-off-by: Rishav Sagar <[email protected]>
…arch-project#5940) This commit applies any feature flags specified by a child class override of the featureFlagSettings() method by default. The result is that test implementations do not have to manually specify the featureFlagSettings when creating test nodes. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]>
The `[Unreleased 3.0]` section of the CHANGELOG on main had lots of entries for items that were backported and released in 2.5. This commit attempts to fix all these issues so that the `[Unreleased 3.0]` section only contains items that have not been backported. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]>
…ject#5907) * Add update settings allowlist for searchable snapshot Allow search related settings to be updated for searchable snapshots. Signed-off-by: Rabi Panda <[email protected]> * Update Changelog Signed-off-by: Rabi Panda <[email protected]> * Add feature flag check Signed-off-by: Rabi Panda <[email protected]> * Use set instead of string arrays for allowed list Signed-off-by: Rabi Panda <[email protected]> * Update Changelog Signed-off-by: Rabi Panda <[email protected]> Signed-off-by: Rabi Panda <[email protected]>
…ility class (opensearch-project#5947) This copies Apache Lucene's o.a.l.util.SetOnce to o.opensearch.common.SetOnce in the opensearch-core library and refactors all opensearch classes from the lucene implementation to the local opensearch implementation. This accomplishes three objectives: 1. remove the dependency on lucene-core library for classes that only import this one lucene utility class, 2. shield opensearch core dependency on this class from any upstream breaking changes, 3. further decouples other foundation classes from third party libraries so they can be refactored from the server module into opensearch support libraries. Signed-off-by: Nicholas Walter Knize <[email protected]>
…st backport (opensearch-project#5960) Signed-off-by: Suraj Singh <[email protected]> Signed-off-by: Suraj Singh <[email protected]>
…pensearch-project#5646) * Replace latches with CompletableFutures for extensions Signed-off-by: Ryan Bogan <[email protected]> * Update Changelog Signed-off-by: Ryan Bogan <[email protected]> * Add one more CompletableFuture complete Signed-off-by: Ryan Bogan <[email protected]> * Changed exception handling Signed-off-by: Ryan Bogan <[email protected]> * Changed exception handling Signed-off-by: Ryan Bogan <[email protected]> * Change exception handling Signed-off-by: Ryan Bogan <[email protected]> * Fix extensions tests Signed-off-by: Ryan Bogan <[email protected]> * Addressed PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Addressed PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Addressed PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Addressed PR Comments Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: Ryan Bogan <[email protected]>
Somehow an entirely misplaced section was put at the top of the file. Additionally two changes had been backported to 2.x but were in the wrong section. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Kunal Kotwani <[email protected]> Signed-off-by: Kunal Kotwani <[email protected]>
) I think this is the last entry that was in the wrong section. It has been [backported to 2.x][1]. [1]: https://github.com/opensearch-project/OpenSearch/blob/4542319c6cf6d6e1136f03f84ec589538daded30/CHANGELOG.md#L26 Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]>
…nsearch-project#5875) * Bump wiremock-jre8-standalone from 2.33.2 to 2.35.0 in /buildSrc Bumps [wiremock-jre8-standalone](https://github.com/wiremock/wiremock) from 2.33.2 to 2.35.0. - [Release notes](https://github.com/wiremock/wiremock/releases) - [Commits](wiremock/wiremock@2.33.2...2.35.0) --- updated-dependencies: - dependency-name: com.github.tomakehurst:wiremock-jre8-standalone dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…-project#5760) * Bump gson from 2.10 to 2.10.1 in /plugins/repository-hdfs Bumps [gson](https://github.com/google/gson) from 2.10 to 2.10.1. - [Release notes](https://github.com/google/gson/releases) - [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md) - [Commits](google/gson@gson-parent-2.10...gson-parent-2.10.1) --- updated-dependencies: - dependency-name: com.google.code.gson:gson dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…#5979) * Create an issue if an on push gradle check fails. Signed-off-by: Marc Handalian <[email protected]> * Add assignees. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>
…arch-project#5972) * Bump json-schema-validator from 1.0.73 to 1.0.76 in /buildSrc Bumps [json-schema-validator](https://github.com/networknt/json-schema-validator) from 1.0.73 to 1.0.76. - [Release notes](https://github.com/networknt/json-schema-validator/releases) - [Changelog](https://github.com/networknt/json-schema-validator/blob/master/CHANGELOG.md) - [Commits](networknt/json-schema-validator@1.0.73...1.0.76) --- updated-dependencies: - dependency-name: com.networknt:json-schema-validator dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <[email protected]>
…oject#5997) This brings our backport workflow script in line with the script from the [tibdex/backport][1] github action implementation. [1]: https://github.com/tibdex/backport/blob/b1bb2aaf611d9f2e5d1900c06493769b9f102a45/.github/workflows/backport.yml#L12-L22 Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Strings.toString is tightly coupled to XContentType.JSON which blocks modularizing foundation classes from the server module to the core library. This change refactors the Strings.toString API to accept a generic MediaType such that the toString implementation detail is encapsulated to the :server module. Signed-off-by: Nicholas Walter Knize <[email protected]>
This commit fixes the changelog entry placement error in opensearch-project#5646 Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Andrew Ross <[email protected]>
…roject#5589) Src files for GeoTile and GeoHash Aggregations on GeoShape with integration tests. Signed-off-by: Navneet Verma <[email protected]>
* Fix flaky SegmentReplicationITs. This change fixes flakiness with segment replication ITs. It does this by updating the wait condition used to ensure replicas are up to date to wait until a searched docCount is reached instead of output of the Segments API that can change if there are concurrent refreshes. It also does this by updating the method used to assert segment stats to wait until the assertion holds true rather than at a point in time. This method is also updated to assert store metadata directly over API output. Signed-off-by: Marc Handalian <[email protected]> * Fix error message to print expected and actual doc counts. Signed-off-by: Marc Handalian <[email protected]> * PR feedback. Signed-off-by: Marc Handalian <[email protected]> * spotless. Signed-off-by: Marc Handalian <[email protected]> Signed-off-by: Marc Handalian <[email protected]>
…earch-project#5970) * Add method to Engine to fetch max seq no of last refresh Co-authored-by: Sachin Kale <[email protected]>
…d index (opensearch-project#5918) * Fix failures in creation and deletion of remote store backed index Signed-off-by: Sachin Kale <[email protected]>
…pensearch-project#6028) OpenSearch uses @opensearch.internal, @opensearch.api, @opensearch.experimental, and feature flags to signal maturity and usage intent of the various implementation components. Each one of these annotations carry a level of backwards compatibility guarantees and support requirements that is not officially documented. This change officially documents those bwc requirements, level of guarantee, and recommended approaches in how to best use the mechanisms to communicate this to the development and user community. Signed-off-by: Nicholas Walter Knize <[email protected]>
…t#5952) Signed-off-by: Rishav Sagar <[email protected]> Co-authored-by: Rishav Sagar <[email protected]>
Signed-off-by: Kunal Kotwani <[email protected]>
Signed-off-by: owaiskazi19 <[email protected]>
…pensearch-project#6797) Signed-off-by: Kunal Kotwani <[email protected]>
* [Segment Replication] Fix resource close handling Signed-off-by: Suraj Singh <[email protected]> * Fix unit tests and change exception type Signed-off-by: Suraj Singh <[email protected]> --------- Signed-off-by: Suraj Singh <[email protected]>
…h-project#6796) Signed-off-by: Kunal Kotwani <[email protected]>
…fault replication type. (opensearch-project#6791) * Add setting to enable segment replication as default replication type. Signed-off-by: Rishikesh1159 <[email protected]> * Fix failing unit tests. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments on PR. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments. Signed-off-by: Rishikesh1159 <[email protected]> * Fix failing unit test. Signed-off-by: Rishikesh1159 <[email protected]> * Address comments. Signed-off-by: Rishikesh1159 <[email protected]> * Refactoring PR. Signed-off-by: Rishikesh1159 <[email protected]> --------- Signed-off-by: Rishikesh1159 <[email protected]>
…pensearch-project#6815) Signed-off-by: Suraj Singh <[email protected]>
…lculations as used in code (opensearch-project#6817) Signed-off-by: Suraj Singh <[email protected]>
…oject#6734) * Add dynamic action registry to ActionModule Signed-off-by: Daniel Widdis <[email protected]> * Update registration of transport actions Signed-off-by: Daniel Widdis <[email protected]> * Generate transport actions dynamically Signed-off-by: Daniel Widdis <[email protected]> * Refactor to combine registry internals Signed-off-by: Daniel Widdis <[email protected]> * Finally figured out the generics (or lack thereof) Signed-off-by: Daniel Widdis <[email protected]> * ExtensionProxyAction is dead! Long live ExtensionAction! Signed-off-by: Daniel Widdis <[email protected]> * Simplify ExtensionTransportActionHandler, fix compile issues Signed-off-by: Daniel Widdis <[email protected]> * Maybe tests will pass with this commit Signed-off-by: Daniel Widdis <[email protected]> * I guess you can't use null as a key in a map Signed-off-by: Daniel Widdis <[email protected]> * Lazy test setup, but this should finally work Signed-off-by: Daniel Widdis <[email protected]> * Add Tests Signed-off-by: Daniel Widdis <[email protected]> * Fix TransportActionRequestFromExtension inheritance Signed-off-by: Daniel Widdis <[email protected]> * Fix return type for transport actions from extensions Signed-off-by: Daniel Widdis <[email protected]> * Fix ParametersInWrongOrderError and add some preemptive null handling Signed-off-by: Daniel Widdis <[email protected]> * NPE is not expected result if params are in correct order Signed-off-by: Daniel Widdis <[email protected]> * Remove redundant class and string parsing, add success boolean Signed-off-by: Daniel Widdis <[email protected]> * Last fix of params out of order. Working test case! Signed-off-by: Daniel Widdis <[email protected]> * Code worked, tests didn't. This is finally done (I think) Signed-off-by: Daniel Widdis <[email protected]> * Add more detail to comments on immutable vs. dynamic maps Signed-off-by: Daniel Widdis <[email protected]> * Add StreamInput getter to ExtensionActionResponse Signed-off-by: Daniel Widdis <[email protected]> * Generalize dynamic action registration Signed-off-by: Daniel Widdis <[email protected]> * Comment and naming fixes Signed-off-by: Daniel Widdis <[email protected]> * Register method renaming Signed-off-by: Daniel Widdis <[email protected]> * Add generic type parameters Signed-off-by: Daniel Widdis <[email protected]> * Improve/simplify which parameter types get passed Signed-off-by: Daniel Widdis <[email protected]> * Revert removal of ProxyAction and changes to transport and requests Signed-off-by: Daniel Widdis <[email protected]> * Wrap ExtensionTransportResponse in a class denoting success Signed-off-by: Daniel Widdis <[email protected]> * Remove generic types as they are incompatible with Guice injection Signed-off-by: Daniel Widdis <[email protected]> * Fix response handling, it works (again) Signed-off-by: Daniel Widdis <[email protected]> * Fix up comments and remove debug logging Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]>
…pensearch-project#6664) Signed-off-by: Sachin Kale <[email protected]>
…in /plugins/repository-azure (opensearch-project#6666) * Bump io.projectreactor.netty:reactor-netty-core Bumps [io.projectreactor.netty:reactor-netty-core](https://github.com/reactor/reactor-netty) from 1.0.24 to 1.1.5. - [Release notes](https://github.com/reactor/reactor-netty/releases) - [Commits](reactor/reactor-netty@v1.0.24...v1.1.5) --- updated-dependencies: - dependency-name: io.projectreactor.netty:reactor-netty-core dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update azure third party audit task Signed-off-by: Suraj Singh <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Suraj Singh <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Suraj Singh <[email protected]>
opensearch-project#6434) * Add wait_for_completion parameter to resize&open&forcemerge APIs (opensearch-project#6228) Signed-off-by: Gao Binlong <[email protected]> * Modify changelog Signed-off-by: Gao Binlong <[email protected]> * fix test failure Signed-off-by: Gao Binlong <[email protected]> * Fix test failure Signed-off-by: Gao Binlong <[email protected]> * change header of new file Signed-off-by: Gao Binlong <[email protected]> * modify changelog Signed-off-by: Gao Binlong <[email protected]> --------- Signed-off-by: Gao Binlong <[email protected]>
…nsearch-project#6838) * [Segment Replication] Fix testAllocationWithDisruption flakyness Signed-off-by: Suraj Singh <[email protected]> * Use more number of nodes with lesser primary shards to decrease chances of primary accumulation on one node Signed-off-by: Suraj Singh <[email protected]> * Update comment Signed-off-by: Suraj Singh <[email protected]> --------- Signed-off-by: Suraj Singh <[email protected]>
…-project#4719) Signed-off-by: Amit Chakraborti <[email protected]>
…earch-project#6424) * Adding numeric optimization support for all numeric types Signed-off-by: gashutos <[email protected]> * modifying CHANGELOG.md Signed-off-by: gashutos <[email protected]> * Handling multi-cluster scenario where SortField serialization was failing Signed-off-by: gashutos <[email protected]> * Fixing javadoc errors Signed-off-by: gashutos <[email protected]> * Fixing nested sort integ tests Signed-off-by: gashutos <[email protected]> * Stremlining behaviour of custom comparator tests too Signed-off-by: gashutos <[email protected]> * Adding more integ tests for IntValuesComparatorSource & fixing few ITs Signed-off-by: gashutos <[email protected]> * Fixing few more integ tests Signed-off-by: gashutos <[email protected]> * Streamlining applySortWidening method with CreateSort and avoid modifying cteated objects of sort Signed-off-by: gashutos <[email protected]> * Correcting licence header Signed-off-by: gashutos <[email protected]> --------- Signed-off-by: gashutos <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]>
…recated setting (opensearch-project#6331) `cluster_manager` role no longer configures the legacy `node.master` setting. Using `node.master: true` configures both cluster_manager and master role to a node which is not correct, only one of them should be assigned Allowing only one of the both roles to be attached instead of both - 'master' which is deprecated - since both the legacy setting `node.master` and `master` role are deprecated - so we need to keep the initial behavior only with deprecated roles and settings. Signed-off-by: Sandesh Kumar <[email protected]>
…search-project#6830) * [SearchableSnapshot] Deleted the cache path on index deletion. Signed-off-by: Harsh Jain <[email protected]> * Added changelog, updated Integ Test. Signed-off-by: Harsh Jain <[email protected]> * Removed changelog entry, updated validation in integ tests. Signed-off-by: Harsh Jain <[email protected]> * Removed unused import statements. Signed-off-by: Harsh Jain <[email protected]> --------- Signed-off-by: Harsh Jain <[email protected]> Co-authored-by: Harsh Jain <[email protected]>
) Signed-off-by: Poojita Raj <[email protected]>
Signed-off-by: Kunal Kotwani <[email protected]>
Signed-off-by: Kunal Kotwani <[email protected]>
The current implementation of the FileCache uses some hand-rolled data structures that would be nice to replace with library implementations. This benchmark will be useful to compare the performance of any future replacements. Sample results (AWS EC2 Ubuntu machine with 32 vCPUs and 64GB of RAM): ``` Benchmark (concurrencyLevel) (maximumNumberOfEntries) Mode Cnt Score Error Units FileCacheBenchmark.get 1 65536 thrpt 2243.092 ops/ms FileCacheBenchmark.get 1 1048576 thrpt 950.818 ops/ms FileCacheBenchmark.get 8 65536 thrpt 5651.150 ops/ms FileCacheBenchmark.get 8 1048576 thrpt 2831.012 ops/ms FileCacheBenchmark.put 1 65536 thrpt 2206.027 ops/ms FileCacheBenchmark.put 1 1048576 thrpt 921.248 ops/ms FileCacheBenchmark.put 8 65536 thrpt 4421.122 ops/ms FileCacheBenchmark.put 8 1048576 thrpt 2624.550 ops/ms FileCacheBenchmark.remove 1 65536 thrpt 12387.999 ops/ms FileCacheBenchmark.remove 1 1048576 thrpt 6324.643 ops/ms FileCacheBenchmark.remove 8 65536 thrpt 22161.031 ops/ms FileCacheBenchmark.remove 8 1048576 thrpt 14826.586 ops/ms FileCacheBenchmark.replace 1 65536 thrpt 2146.572 ops/ms FileCacheBenchmark.replace 1 1048576 thrpt 947.612 ops/ms FileCacheBenchmark.replace 8 65536 thrpt 4405.339 ops/ms FileCacheBenchmark.replace 8 1048576 thrpt 2707.204 ops/ms ``` Signed-off-by: Andrew Ross <[email protected]>
…opensearch-project#6889) Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Kunal Kotwani <[email protected]>
) Signed-off-by: Varun Bansal <[email protected]> (cherry picked from commit d8d6e73)
Gradle Check (Jenkins) Run Completed with:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Backport #6834
Issues Resolved
NA
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.