From c1d30f01314611d8a771e6f9dd52a59f083924a7 Mon Sep 17 00:00:00 2001 From: Davis Plumlee <56367316+dplumlee@users.noreply.github.com> Date: Mon, 12 Feb 2024 12:57:15 -0500 Subject: [PATCH] [Security Solution] Per-field diffs test plan (#176474) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Resolves https://github.com/elastic/kibana/issues/176473 This PR introduces a test plan for the per-field diff preview. This preview is displayed in the upgrade prebuilt rule flyout under the `Updates` tab. Screenshot 2024-02-08 at 1 35 05 AM --- .../installation_and_upgrade.md | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md index b60609b45be9d..41e379906eb42 100644 --- a/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md +++ b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md @@ -64,6 +64,11 @@ Status: `in progress`. The current test plan matches `Milestone 2` of the [Rule - [**Scenario: Properties with semantically equal values should not be shown as modified**](#scenario-properties-with-semantically-equal-values-should-not-be-shown-as-modified) - [**Scenario: Unchanged sections of a rule should be hidden by default**](#scenario-unchanged-sections-of-a-rule-should-be-hidden-by-default) - [**Scenario: Properties should be sorted alphabetically**](#scenario-properties-should-be-sorted-alphabetically) + - [Rule upgrade workflow: viewing rule changes in per-field diff view](#rule-upgrade-workflow-viewing-rule-changes-in-per-field-diff-view) + - [**Scenario: User can see changes in a side-by-side per-field diff view**](#scenario-user-can-see-changes-in-a-side-by-side-per-field-diff-view) + - [**Scenario: Field groupings should be rendered together in the same accordion panel**](#scenario-field-groupings-should-be-rendered-together-in-the-same-accordion-panel) + - [**Scenario: Undefined values are displayed with empty diffs**](#scenario-undefined-values-are-displayed-with-empty-diffs) + - [**Scenario: Field diff components have the same grouping and order as in rule details overview**](#scenario-field-diff-components-have-the-same-grouping-and-order-as-in-rule-details-overview) - [Rule upgrade workflow: preserving rule bound data](#rule-upgrade-workflow-preserving-rule-bound-data) - [**Scenario: Rule bound data is preserved after upgrading a rule to a newer version with the same rule type**](#scenario-rule-bound-data-is-preserved-after-upgrading-a-rule-to-a-newer-version-with-the-same-rule-type) - [**Scenario: Rule bound data is preserved after upgrading a rule to a newer version with a different rule type**](#scenario-rule-bound-data-is-preserved-after-upgrading-a-rule-to-a-newer-version-with-a-different-rule-type) @@ -953,6 +958,99 @@ When a user expands all hidden sections Then all properties of the rule should be sorted alphabetically ``` +### Rule upgrade workflow: viewing rule changes in per-field diff view + +#### **Scenario: User can see changes in a side-by-side per-field diff view** + +**Automation**: 1 e2e test + +```Gherkin +Given X prebuilt rules are installed in Kibana +And for Y of these rules new versions are available +When user opens the Rule Updates table and selects a rule +Then the per-field upgrade preview should open +And rule changes should be displayed in a two-column diff view with each field in its own accordion component +And all field diff accordions should be open by default +And correct rule version numbers should be displayed in their respective columns +When the user selects another rule without closing the preview +Then the preview should display the changes for the newly selected rule +``` + +#### **Scenario: User can see changes when updated rule is a different rule type** + +**Automation**: 1 UI integration test + +```Gherkin +Given a prebuilt rule is installed in Kibana +And this rule has an update available that changes the rule type +When user opens the upgrade preview +Then the rule type changes should be displayed in grouped field diffs with corresponding query fields +And a tooltip is displayed with information about changing rule types +``` + +#### **Scenario: Field groupings should be rendered together in the same accordion panel** + +**Automation**: 1 UI integration test + +```Gherkin +Given a prebuilt rule is installed in Kibana +And this rule contains one or more values +When user opens the upgrade preview +The diff accordion panel should display its grouped rule properties +And each property should have its name displayed inside the panel above its value + +Examples: +| field | +| data_source | +| kql_query | +| eql_query | +| esql_query | +| threat_query | +| rule_schedule | +| rule_name_override | +| timestamp_override | +| timeline_template | +| building_block | +| threshold | +``` + +#### **Scenario: Undefined values are displayed with empty diffs** + +**Automation**: 1 UI integration test + +```Gherkin +Given a prebuilt rule is installed in Kibana +And this rule has field in the version that didn't exist in the version +When a user opens the upgrade preview +Then the preview should open +And the old/new field should render an empty panel + +Examples: +| version_one | version_two | +| target | current | +| current | target | +``` + +#### **Scenario: Field diff components have the same grouping and order as in rule details overview** + +**Automation**: 1 UI integration test + +```Gherkin +Given a prebuilt rule is installed in Kibana +And this rule has multiple fields that are different between the current and target version +When a user opens the upgrade preview +Then the multiple field diff accordions should be sorted in the same order as on the rule details overview tab +And the field diff accordions should be grouped inside its corresponding
accordion +And any
accordion that doesn't have fields inside it shouldn't be displayed + +Examples: +| section | +| About | +| Definition | +| Schedule | +| Setup Guide | +``` + ### Rule upgrade workflow: preserving rule bound data #### **Scenario: Rule bound data is preserved after upgrading a rule to a newer version with the same rule type**