[7.x] [Security Solution] [Detections] Improves custom query rule upgrade test (#114454)

[MadameSheema]

Backports the following commits to 7.x:

• [Security Solution] [Detections] Improves custom query rule upgrade test ([Security Solution] [Detections] Improves custom query rule upgrade test #114454)

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request

• Commit: 8572c81

• Storybooks Preview

• Flaky suites:

  • xpack-kibana-ciGroup8

---

Test Failures

Kibana Pipeline / general / Performance Tests.x-pack/test/performance/tests/reporting_dashboard·ts.performance reporting dashbaord downloaded PDF has OK status

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 3 times on tracked branches: https://github.com/elastic/kibana/issues/110470

[00:00:00]     │
[00:00:00]       └-: performance
[00:00:00]         └-> "before all" hook in "performance"
[00:00:00]         └-: reporting dashbaord
[00:00:00]           └-> "before all" hook for "downloaded PDF has OK status"
[00:00:00]           └-> "before all" hook for "downloaded PDF has OK status"
[00:00:00]             │ debg resolved import for x-pack/test/performance/kbn_archives/reporting_dashboard to /dev/shm/workspace/parallel/19/kibana/x-pack/test/performance/kbn_archives/reporting_dashboard.json
[00:00:00]             │ info importing 4 saved objects { space: undefined }
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/KI-fgKqAS9S1feLd2Qh9DQ] update_mapping [_doc]
[00:00:00]             │ info [o.e.x.i.IndexLifecycleTransition] [node-01] moving index [.ds-ilm-history-5-2021.10.13-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] in policy [ilm-history-ilm-policy]
[00:00:00]             │ info [o.e.x.i.IndexLifecycleTransition] [node-01] moving index [.ds-ilm-history-5-2021.10.13-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [ilm-history-ilm-policy]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/KI-fgKqAS9S1feLd2Qh9DQ] update_mapping [_doc]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/KI-fgKqAS9S1feLd2Qh9DQ] update_mapping [_doc]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/KI-fgKqAS9S1feLd2Qh9DQ] update_mapping [_doc]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/KI-fgKqAS9S1feLd2Qh9DQ] update_mapping [_doc]
[00:00:00]             │ succ import success
[00:00:00]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Loading "mappings.json"
[00:00:00]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Loading "data.json.gz"
[00:00:00]             │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [foo] creating index, cause [api], templates [], shards [1]/[1]
[00:00:00]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Created index "foo"
[00:00:00]             │ debg [x-pack/test/performance/es_archives/reporting_dashboard] "foo" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:00:02]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Indexed 10000 docs into "foo"
[00:00:02]           └-> downloaded PDF has OK status
[00:00:02]             └-> "before each" hook: global before each for "downloaded PDF has OK status"
[00:00:02]             │ debg navigating to dashboards url: http://localhost:61191/app/dashboards
[00:00:02]             │ debg navigate to: http://localhost:61191/app/dashboards
[00:00:02]             │ debg browser[INFO] http://localhost:61191/login?next=%2Fapp%2Fdashboards%3F_t%3D1634137485175 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:02]             │
[00:00:02]             │ debg browser[INFO] http://localhost:61191/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:02]             │ debg ... sleep(700) start
[00:00:03]             │ debg ... sleep(700) end
[00:00:03]             │ debg returned from get, calling refresh
[00:00:04]             │ debg browser[INFO] http://localhost:61191/login?next=%2Fapp%2Fdashboards%3F_t%3D1634137485175 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:04]             │
[00:00:04]             │ debg browser[INFO] http://localhost:61191/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:05]             │ debg currentUrl = http://localhost:61191/login?next=%2Fapp%2Fdashboards%3F_t%3D1634137485175
[00:00:05]             │          appUrl = http://localhost:61191/app/dashboards
[00:00:05]             │ debg TestSubjects.find(kibanaChrome)
[00:00:05]             │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:05]             │ debg Found login page
[00:00:05]             │ debg TestSubjects.setValue(loginUsername, test_user)
[00:00:05]             │ debg TestSubjects.click(loginUsername)
[00:00:05]             │ debg Find.clickByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:05]             │ debg Find.findByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:05]             │ERROR browser[SEVERE] http://localhost:61191/api/licensing/info - Failed to load resource: the server responded with a status of 401 (Unauthorized)
[00:00:05]             │ debg TestSubjects.setValue(loginPassword, changeme)
[00:00:05]             │ debg TestSubjects.click(loginPassword)
[00:00:05]             │ debg Find.clickByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:05]             │ debg Find.findByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:06]             │ debg TestSubjects.click(loginSubmit)
[00:00:06]             │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:06]             │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:06]             │ debg Find.waitForDeletedByCssSelector('.kibanaWelcomeLogo') with timeout=10000
[00:00:06]             │ proc [kibana]   log   [15:04:48.845] [info][plugins][routes][security] Logging in with provider "basic" (basic)
[00:00:06]             │ debg browser[INFO] http://localhost:61191/app/dashboards?_t=1634137485175 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:06]             │
[00:00:06]             │ debg browser[INFO] http://localhost:61191/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:06]             │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:08]             │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"] nav:not(.ng-hide)') with timeout=60000
[00:00:09]             │ debg browser[INFO] http://localhost:61191/app/dashboards?_t=1634137491359 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:09]             │
[00:00:09]             │ debg browser[INFO] http://localhost:61191/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:09]             │ debg Finished login process currentUrl = http://localhost:61191/app/dashboards
[00:00:09]             │ debg ... sleep(501) start
[00:00:10]             │ debg ... sleep(501) end
[00:00:10]             │ debg in navigateTo url = http://localhost:61191/app/dashboards
[00:00:10]             │ debg Waiting up to 20000ms for dashboard landing page...
[00:00:10]             │ debg onDashboardLandingPage
[00:00:10]             │ debg TestSubjects.exists(dashboardLandingPage)
[00:00:10]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=5000
[00:00:10]             │ debg Load Saved Dashboard dashboard
[00:00:10]             │ debg gotoDashboardLandingPage
[00:00:10]             │ debg onDashboardLandingPage
[00:00:10]             │ debg TestSubjects.exists(dashboardLandingPage)
[00:00:10]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=5000
[00:00:10]             │ debg searchForItemWithName: dashboard
[00:00:10]             │ debg TestSubjects.find(tableListSearchBox)
[00:00:10]             │ debg Find.findByCssSelector('[data-test-subj="tableListSearchBox"]') with timeout=10000
[00:00:11]             │ debg isGlobalLoadingIndicatorVisible
[00:00:11]             │ debg TestSubjects.exists(globalLoadingIndicator)
[00:00:11]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:00:11]             │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:00:11]             │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:00:13]             │ debg TestSubjects.click(dashboardListingTitleLink-dashboard)
[00:00:13]             │ debg Find.clickByCssSelector('[data-test-subj="dashboardListingTitleLink-dashboard"]') with timeout=10000
[00:00:13]             │ debg Find.findByCssSelector('[data-test-subj="dashboardListingTitleLink-dashboard"]') with timeout=10000
[00:00:14]             │ debg isGlobalLoadingIndicatorVisible
[00:00:14]             │ debg TestSubjects.exists(globalLoadingIndicator)
[00:00:14]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:00:14]             │ debg browser[INFO] http://localhost:61191/app/dashboards#/view/37b49c50-2dc6-11eb-8af3-cb3aa84dbabd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now)) 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:14]             │
[00:00:14]             │ debg browser[INFO] http://localhost:61191/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:15]             │ debg --- retry.tryForTime error: [data-test-subj="globalLoadingIndicator"] is not displayed
[00:00:16]             │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:00:16]             │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:00:16]             │ debg TestSubjects.missingOrFail(dashboardLandingPage)
[00:00:16]             │ debg Find.waitForDeletedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=10000
[00:00:16]             │ debg openPdfReportingPanel
[00:00:16]             │ debg openShareMenuItem title:PDF Reports
[00:00:16]             │ debg TestSubjects.exists(shareContextMenu)
[00:00:16]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="shareContextMenu"]') with timeout=2500
[00:00:17]             │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [.async-search] creating index, cause [api], templates [], shards [1]/[0]
[00:00:19]             │ debg --- retry.tryForTime error: [data-test-subj="shareContextMenu"] is not displayed
[00:00:19]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/KI-fgKqAS9S1feLd2Qh9DQ] update_mapping [_doc]
[00:00:19]             │ debg TestSubjects.click(shareTopNavButton)
[00:00:19]             │ debg Find.clickByCssSelector('[data-test-subj="shareTopNavButton"]') with timeout=10000
[00:00:19]             │ debg Find.findByCssSelector('[data-test-subj="shareTopNavButton"]') with timeout=10000
[00:00:20]             │ debg Find.findByCssSelector('div.euiContextMenuPanel') with timeout=10000
[00:00:21]             │ debg TestSubjects.click(sharePanel-PDFReports)
[00:00:21]             │ debg Find.clickByCssSelector('[data-test-subj="sharePanel-PDFReports"]') with timeout=10000
[00:00:21]             │ debg Find.findByCssSelector('[data-test-subj="sharePanel-PDFReports"]') with timeout=10000
[00:00:22]             │ debg Find.waitForElementStale with timeout=10000
[00:00:23]             │ debg TestSubjects.click(generateReportButton)
[00:00:23]             │ debg Find.clickByCssSelector('[data-test-subj="generateReportButton"]') with timeout=10000
[00:00:23]             │ debg Find.findByCssSelector('[data-test-subj="generateReportButton"]') with timeout=10000
[00:00:23]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.16.0_001/KI-fgKqAS9S1feLd2Qh9DQ] update_mapping [_doc]
[00:00:24]             │ debg getReportURL
[00:00:24]             │ debg TestSubjects.getAttribute(downloadCompletedReportButton, href, tryTimeout=120000, findTimeout=60000)
[00:00:24]             │ debg TestSubjects.find(downloadCompletedReportButton)
[00:00:24]             │ debg Find.findByCssSelector('[data-test-subj="downloadCompletedReportButton"]') with timeout=60000
[00:01:25]             │ debg --- retry.tryForTime error: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:01:25]             │      Wait timed out after 61154ms
[00:01:25]             │ debg TestSubjects.find(downloadCompletedReportButton)
[00:01:25]             │ debg Find.findByCssSelector('[data-test-subj="downloadCompletedReportButton"]') with timeout=60000
[00:02:26]             │ debg --- retry.tryForTime error: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:02:26]             │      Wait timed out after 61107ms
[00:02:27]             │ info Taking screenshot "/dev/shm/workspace/parallel/19/kibana/x-pack/test/functional/screenshots/failure/performance reporting dashbaord downloaded PDF has OK status.png"
[00:02:27]             │ info Current URL is: http://localhost:61191/app/dashboards#/view/37b49c50-2dc6-11eb-8af3-cb3aa84dbabd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:02:27]             │ info Saving page source to: /dev/shm/workspace/parallel/19/kibana/x-pack/test/performance/failure_debug/html/performance reporting dashbaord downloaded PDF has OK status.html
[00:02:27]             └- ✖ fail: performance reporting dashbaord downloaded PDF has OK status
[00:02:27]             │      Error: retry.tryForTime timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:02:27]             │ Wait timed out after 61107ms
[00:02:27]             │     at /dev/shm/workspace/parallel/19/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:02:27]             │     at runMicrotasks (<anonymous>)
[00:02:27]             │     at processTicksAndRejections (internal/process/task_queues.js:95:5)
[00:02:27]             │       at onFailure (/dev/shm/workspace/parallel/19/kibana/test/common/services/retry/retry_for_success.ts:17:9)
[00:02:27]             │       at retryForSuccess (/dev/shm/workspace/parallel/19/kibana/test/common/services/retry/retry_for_success.ts:59:13)
[00:02:27]             │       at RetryService.tryForTime (/dev/shm/workspace/parallel/19/kibana/test/common/services/retry/retry.ts:22:12)
[00:02:27]             │       at TestSubjects.getAttribute (/dev/shm/workspace/parallel/19/kibana/test/functional/services/common/test_subjects.ts:190:12)
[00:02:27]             │       at ReportingPageObject.getReportURL (test/functional/page_objects/reporting_page.ts:36:17)
[00:02:27]             │       at Context.<anonymous> (test/performance/tests/reporting_dashboard.ts:50:7)
[00:02:27]             │       at Object.apply (/dev/shm/workspace/parallel/19/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:02:27]             │ 
[00:02:27]             │ 

Stack Trace

Error: retry.tryForTime timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
Wait timed out after 61107ms
    at /dev/shm/workspace/parallel/19/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at onFailure (/dev/shm/workspace/parallel/19/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/dev/shm/workspace/parallel/19/kibana/test/common/services/retry/retry_for_success.ts:59:13)
    at RetryService.tryForTime (/dev/shm/workspace/parallel/19/kibana/test/common/services/retry/retry.ts:22:12)
    at TestSubjects.getAttribute (/dev/shm/workspace/parallel/19/kibana/test/functional/services/common/test_subjects.ts:190:12)
    at ReportingPageObject.getReportURL (test/functional/page_objects/reporting_page.ts:36:17)
    at Context.<anonymous> (test/performance/tests/reporting_dashboard.ts:50:7)
    at Object.apply (/dev/shm/workspace/parallel/19/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

---

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream