Removing default integtest.sh (1.1). #219
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Peter Zhu <[email protected]>
Signed-off-by: Tyler Ohlsen <[email protected]>
Signed-off-by: Tyler Ohlsen <[email protected]>
Signed-off-by: Tyler Ohlsen <[email protected]>
Signed-off-by: Tyler Ohlsen <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>
…nsearch-integtest-demo Add Integtest.sh for OpenSearch integtest setups
…ject#121) This PR is a conglomerate of the following PRs. opensearch-project#60 opensearch-project#64 opensearch-project#65 opensearch-project#67 opensearch-project#68 opensearch-project#69 opensearch-project#70 opensearch-project#71 opensearch-project#74 opensearch-project#75 opensearch-project#76 opensearch-project#77 opensearch-project#78 opensearch-project#79 opensearch-project#82 opensearch-project#83 opensearch-project#84 opensearch-project#92 opensearch-project#94 opensearch-project#93 opensearch-project#95 kaituo#1 kaituo#2 kaituo#3 kaituo#4 kaituo#5 kaituo#6 kaituo#7 kaituo#8 kaituo#9 kaituo#10 This spreadsheet contains the mappings from files to PR number (bug fix in my AD fork and tests are not included): https://gist.github.com/kaituo/9e1592c4ac4f2f449356cb93d0591167
* don't replace detector user when update Signed-off-by: Yaliang Wu <[email protected]> * fix wrong doc link
opensearch-project#128) * Improve preview and exception handling in AnomalyResultTransportAction This PR commits changes from the following PRs: kaituo#11 kaituo#12
* tune data model for unified flow Signed-off-by: Yaliang Wu <[email protected]> * support unified flow for single entity detector Signed-off-by: Yaliang Wu <[email protected]> * add realtime task Signed-off-by: Yaliang Wu <[email protected]> * support preview with detector directly Signed-off-by: Yaliang Wu <[email protected]> * support historical analysis for HC detector (opensearch-project#104) * support run historical analysis for HC detector Signed-off-by: Yaliang Wu <[email protected]> * add more javadoc Signed-off-by: Yaliang Wu <[email protected]> * address comments: rename method, remove comments etc Signed-off-by: Yaliang Wu <[email protected]> * add more comments for task action Signed-off-by: Yaliang Wu <[email protected]> * address comments from old CR: mainly renaming and add more comments Signed-off-by: Yaliang Wu <[email protected]> Co-authored-by: Yaliang Wu <[email protected]> * support HC task profiles in profile API;error handling (opensearch-project#123) * support HC task profiles in profile API;error handling Signed-off-by: Yaliang Wu <[email protected]> * add init progress of realtime task (opensearch-project#130) * add init progress of realtime task * add realtime task cache * remove DetectionStateHandler, will track error in realtime task * add more comments * add delete anomaly results API (opensearch-project#131) Signed-off-by: Yaliang Wu <[email protected]> * maintain running historical task in hourly cron (opensearch-project#132) * maintain running historical task in hourly cron Signed-off-by: Yaliang Wu <[email protected]> * maintain realtime tasks * address comments Co-authored-by: Yaliang Wu <[email protected]>
* compact rcf
… fix hard code query size of historical analysis (opensearch-project#150) * avoid sending back verbose error message and wrong 500 error to user Signed-off-by: Yaliang Wu <[email protected]> * put general error message into constants
Signed-off-by: lai <[email protected]>
…t#162) Disable model splitting in single-stream detectors We split and distribute models to different nodes to avoid large models on a single node. The splitting is unnecessary after introducing compact rcf as the model is smaller (at least 4x smaller). Splitting also undoes the shared point store optimization among trees. Also, splitting brings complications when computing expected values. Thus, this PR disables splitting by increasing the desired model size. We won't split a model whose size is less than the desired size. This PR adjusts max features and shingle size accordingly to avoid huge models without explicit benefits. This PR also adjusts the model size formula due to the change aws/random-cut-forest-by-aws#265. I will update the rcf version once rcf 2.0 is released in maven. Testing done: 1. tested single-stream models won't be split after the change. 2. Updated unit tests.
enable junit 5
* Bug fixes and disable BWC This PR did the following: 1)Fixed a bug in ModelPartitioner that we should use estimated rcf instead of total model size (including rcf and threshold models). 2)Disabled backward-compatibility check so that we can run multi-category detectors in OpenSearch 1.0. 3)remove unused fields in MemoryTracker 4)add null check when deserializing the sample vector Testing done: 1)added unit tests 2)did e2e tests for the basic single-stream detector workflow.
…oject#157) * Handle more AD exceptions thrown over the wire OpenSearch restricts the kind of exceptions that can be thrown over the wire (Read OpenSearchException.OpenSearchExceptionHandle). Since we cannot add our exception like ResourceNotFoundException without modifying OpenSearch's code, we must unwrap the NotSerializableExceptionWrapper and check its root cause message. This PR adds checks all AD exceptions in such cases. Previously, we only checked a couple of them. Previously, we wrap all exceptions thrown in AD using one of AnomalyDetectionExceptions. The wrap brings a complication to NotSerializableExceptionWrapper thrown over the wire as NotSerializableExceptionWrapper won't keep the original AnomalyDetectionExceptions object and only keeps the cause message. Therefore, we won't be able to restore the original exception encapsulated inside AnomalyDetectionExceptions. This PR stops wrapping the original exception inside AnomalyDetectionExceptions. This PR also adds a null check inside EntityModel in case of a potential null pointer exception. Testing done: 1. Added unit tests to check if exceptions wrapped inside NotSerializableExceptionWrapper can be decoded. 2. Did e2e testing for basic single-stream and HCAD workflow.
…nsearch-project#158) * Adding mixed cluster bwc tests Signed-off-by: Vacha <[email protected]> * Adding anomaly detector assertions for bwc Signed-off-by: Vacha <[email protected]> * Adding rolling upgrade and full restart upgrade bwc tests Signed-off-by: Vacha <[email protected]>
Signed-off-by: Miki <[email protected]>
…#159) * support historical analysis for multi-category HC Signed-off-by: Yaliang Wu <[email protected]> * add todo for tuning timeout * add delete AD result setting; tune minimum doc count for historical analysis * fix minmum doc count for historical analysis * fix wrong min doc threshold
* Adding support for BWC tests to run in CI Signed-off-by: Vacha <[email protected]> * Adding bwc test commands in developer guide Signed-off-by: Vacha <[email protected]>
First, this PR adds an empty string check in PriorityCache. Second, in EntityColdStarter, this PR clears samples after using them for training. Third, this PR removes unused fields and local variables in ModelManager and BulkUtil. Fourth, this PR decreases the concurrent preview from 5 to 2, with a conservative purpose to reduce potential GC. Fifth, this PR finishes unit tests owed due to my previous PRs. 99% code of the PR is about tests. Testing done: 1. Added unit tests. 2. made sure the basic e2e workflow still works.
Signed-off-by: Vacha <[email protected]>
…roject#182) * Limit the max models shown on the stats and profile API Previously, the number of models we show on the stats/profile API is unbounded. The unbounded models can cause potential memory/network issues if a vast number of models exist. This PR sets an upper bound and makes the bound a dynamic setting. Due to the above change, the models may not be complete, and thus I added a model_count to the stats/profile API to show the actual models present. This PR also fixed bugs: First, AnomalyDetectorProfileRunner used a listener instead of delegateListener, which can cause the profile API to return early before collecting all outputs. Second, I forgot to support the multi-category fields in AnomalyDetectorProfileRunner.profileEntityStats. This PR adds support there. In detail, profile API needs to report total entities. We can use cardinality aggregation for a single-category field. But we cannot do that for multi-category fields as it requires scripting to generate run time fields, which is expensive. We work around the problem by using a composite query to find the first 10_000 buckets. Generally, traversing all buckets/combinations can't be done without visiting all matches, which is costly for data with many entities. Given that it is often enough to have a lower bound of the number of entities, such as "there are at least 10000 entities", the default is set to 10,000. That is, requests will count the total entities up to 10,000. Third, I fixed a potential null pointer exception in ADTaskManager. Testing done: 1. e2e workflow tests by verifying profile/stats API shows correct output, and the limit is in effect. 2. backward compatibility tests to make sure the newly added model_count does not cause issues. 3. added unit tests.
Previously, we hardcoded HCAD (including real-time and historical) shingle size to be one due to time constraints when releasing HCAD. This PR enabled shingle. By default, the shingle size is eight but can be customized. I removed shingling-related code (even though I didn't allow shingling, but there is some half-finished shingling code) in HCAD as RCF can create shingles by itself (turned on via the internalShinglingEnabled configuration). This PR also: 1. upgrade RCF to 2.0.1. 2. Removed MULTI_ENTITY_NUM_TREES with NUM_TREES as both HCAD and single-stream detectors have the same number of trees. 3.Fixed a bug in ModelManager.score method: I should compute anomaly grade and confidence before updating rcf and threshold model. Testing done: 1. added unit tests for the bug found. 2. Manual testing to verify HCAD single stream and historical workflow still work, and the RCF models have desired shingle size.
…sk (opensearch-project#170) * support backward compatibility of historical analysis and realtime task * scale task lane automatically * fix reset task state and update task * tune scale task slots logic and move it to task manager * use AD version hash ring for realtime job; reserve 1 task slot when scale down * rollback single flow task type as single entity to save frontend effort * remove request id from cron request; address comments * support BWC of RCF result response based on AD version * remove BWC check for anomaly result response * address comments: fix log; add more comments * enhance stop logic;address comments * add more comments;tune function name * add AD version hash ring with cooldown period for realtime job * tune hash ring * tune hash ring;add more ut * use new PluginInfo constrctor as it's changed in OpenSearch PR#848 * tune error message of null AD version
* block updating detector category field Signed-off-by: Yaliang Wu <[email protected]> * remove debug code * ignore category field order when compare
…earch-project#189) * Adding support to run BWC tests for nightly and release builds Signed-off-by: Sarat Vemulapalli <[email protected]> * Updating test suite for AD Signed-off-by: Sarat Vemulapalli <[email protected]>
…cal analysis (opensearch-project#197) Signed-off-by: Yaliang Wu <[email protected]>
Signed-off-by: Tyler Ohlsen <[email protected]>
* reset realtime task state Signed-off-by: Yaliang Wu <[email protected]> * try to delete AD tasks and job when detector not found * address comments * add more comments * fix bugs; tune max running entities setting; add more UT * rename variable * add more comment
…pensearch-project#202) * Fix flaky test and log level/messages and enable auto-expand replica of AD job index This PR (hopefully, as I cannot reproduce the failure locally) fixed flaky tests in MultiEntityResultTests. The tests are flaky, maybe because we expect two pages in our pagination, but we may create more than two pages due to a race condition. Please read comments in MultiEntityResultTests for detail. This PR also changes the log level of the updating real-time task log from info to debug. We don't need info as Opensearch prints the log repeatedly in each interval. Also, I changed the log message in ADTaskManager to match what the relevant code does. This PR also enables auto-expand replication for AD job indexes. The job scheduler puts both primary and replica shards in the hash ring. Enabling auto-expand the number of replicas based on the number of data nodes (up to 20) in the cluster so that each node can become a coordinating node. Enabling auto-expanding is useful when customers scale out their cluster so that we can do adaptive scaling accordingly. Also, this PR changed the primary number of shards of the AD job index to 1 as the AD job index is small. Testing done: 1. Checked that the AD job index setting change is effective and won't negatively impact normal e2e workflow.
Signed-off-by: dblock <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Coming from opensearch-project/opensearch-build#497, removing default integtest.sh.