[8.x] [Security Solution] Implement concurrency control for Prebuilt Upgrade workflow (#203604) #205877
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e workflow (elastic#203604) **Resolves:** elastic#200134 ## Summary This PR implements concurrency control to make sure user has the recent rule updates data in Rule Upgrade flyout. Any modifications saved in Rule Upgrade flyout are reset upon new `revision` or `version` detected. ## Details Concurrency control is important to provide better UX. Multiple users work in Kibana in parallel and new prebuilt rules package version can be released in any time. Attempts to upgrade a rule with outdated `revision` and/or `version` results in failed request. Users may experience multiple rule upgrade failure in that case causing a lot of confusion. More experienced users may guess to reload the page to continue. Typical reasons leading to `revision` and/or `version` change are the following - Current rule has been edited will bump rule's `revision`. For example the rule currently shown in Rule Upgrade flyout has been edited by someone else. - Prebuilt rules package got released will give provide rule assets with higher `version`. Rules having upgrades in the currently installed package and in a new one are affected. This PR mitigates the described issues by implementing concurrency control. It sets up `_review` API endpoint refetch interval to 5 minutes to fetch fresh data. In case a higher `revision` or `version` is detected for some rule this rule's resolved conflicts and customizations performed in Rule Upgrade flyout get cleared. ## Screenshots - `revision` change (refresh interval was reduced to 30 seconds to make the video shorter) https://github.com/user-attachments/assets/98d2a22f-9338-482a-a7b2-1e170b9642ce - `version` change (refresh interval was reduced to 1 minute to make the video shorter) https://github.com/user-attachments/assets/2b7c23f0-5a50-471e-aa7f-8d9b2aecc957 ## How to test locally There are two cases for testing - `revision` change - `version` change ### Test `revision` change Revision change means the rule has been edited. Use the following steps to test it - Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled - Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml` - Clear Elasticsearch data - Run Elasticsearch and Kibana locally (do not open Kibana in a web browser) - Install an outdated version of the `security_detection_engine` Fleet package ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1 ``` - Install prebuilt rules ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform ``` - Open `Detection Rules (SIEM)` Page -> `Rule Updates` - Open Rule upgrade flyout for some rule - Make changes to rule field(s) and save them (do not upgrade the rule) - Open the other web browser tab with Kibana - Navigate to the same rule's editing page - Change any field and save the changes - Return back to the first tab and wait for data to be refetched (data refresh interval is 5 minutes, wait for `_review` request in the Dev Tool's Network tab) - Make sure the changes you made for field(s) got reverted ### Test `version` change Version change means a new package version was released. Do the following to test it - Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled - Allow internal APIs via adding `server.restrictInternalApis: false` to `kibana.dev.yaml` - Clear Elasticsearch data - Run Elasticsearch and Kibana locally (do not open Kibana in a web browser) - Set `xpack.securitySolution.prebuiltRulesPackageVersion: 8.15.2` in `kibana.dev.yaml` - Install an outdated version of the `security_detection_engine` Fleet package ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1 ``` - Install prebuilt rules ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform ``` - Open `Detection Rules (SIEM)` Page -> `Rule Updates` - Open Rule upgrade flyout for a rule having updates in packages `v8.15.2` and `.8.17.1-beta.1` for example `Suspicious Web Browser Sensitive File Access` - Make changes to rule field(s) and save them (do not upgrade the rule) - Set `xpack.securitySolution.prebuiltRulesPackageVersion: 8.17.1-beta.1` in `kibana.dev.yaml` - Open the other web browser tab with Kibana - Navigate to Security Solution plugin to install the OR install the package `8.17.1-beta.1` via API request ```bash curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.17.1-beta.1 ``` - Return back to the first tab and wait for data to be refetched (data refresh interval is 5 minutes, wait for `_review` request in the Dev Tool's Network tab) - Make sure the changes you made for field(s) got the recent target rule values Alternatively you can spin up EPR locally and publish package updates with rule's version bumped. (cherry picked from commit 1929279)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.x:Questions ?
Please refer to the Backport tool documentation