[Synthetics] chore - synthetics unskip test in mki #204273

dominiqueclarke · 2024-12-13T19:00:01Z

Summary

Unskip test that was previously failing in MKI.

These tests have been run against MKI.

Background on the test

The test confirms that synthetics integration policies auto update when the package version is updated. It attempts to install an older version of the package and then run fleet setup to install a newer version of the package and trigger auto upgrade

Why it failed

The test expected a lower version of the package to be installed, but a more recent version was already installed. Therefore, when we went to create synthetics integration policies they were installed with the most recent version.

How we fixed it

Before this test, we now uninstall the Synthetics integration to ensure we can force install an earlier version.

…ynthetics-unskip-test-in-mki

elasticmachine · 2024-12-16T15:19:05Z

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

dominiqueclarke · 2024-12-16T15:19:35Z

...gration/deployment_agnostic/apis/observability/synthetics/create_monitor_private_location.ts

-    //     await security.role.delete(roleName);
-    //   }
-    // });
+    it('handles spaces', async () => {


This test was accidentally commented out. I restored it.

maryam-saeidi · 2024-12-16T16:47:16Z

@dominiqueclarke Does it make sense to run the flaky test runner for this test?

dominiqueclarke · 2024-12-16T17:02:17Z

@dominiqueclarke Does it make sense to run the flaky test runner for this test?

Does flaky test runner integrate with mki in any way?

maryam-saeidi · 2024-12-17T11:34:27Z

Does flaky test runner integrate with mki in any way?

Not that I am aware of. I meant verifying the changes are not flaky outside of the MKI environment.

wayneseymour · 2024-12-18T11:05:48Z

Adding a link for the issue this address: #204204

wayneseymour

I ran this successfully against an oblt instance from my local! Looks good!

wayneseymour · 2024-12-18T10:52:43Z

...gration/deployment_agnostic/apis/observability/synthetics/create_monitor_private_location.ts

 import { addMonitorAPIHelper, keyToOmitList, omitMonitorKeys } from './create_monitor';
 import { SyntheticsMonitorTestService } from '../../../services/synthetics_monitor';

 export default function ({ getService }: DeploymentAgnosticFtrProviderContext) {
  describe('PrivateLocationAddMonitor', function () {
-    // see details: https://github.com/elastic/kibana/issues/204204
-    this.tags(['failsOnMKI']);
    const kibanaServer = getService('kibanaServer');
    const supertestAPI = getService('supertestWithoutAuth');


Suggested change

const supertestAPI = getService('supertestWithoutAuth');

const supertestWithoutAuth = getService('supertestWithoutAuth');

I recognize this line is not within your pr's changes, but this is bad practice.

kibanamachine · 2025-01-02T16:59:53Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7639

[✅] x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts: 50/50 tests passed.

see run history

elasticmachine · 2025-01-02T21:00:28Z

💚 Build Succeeded

Buildkite Build
Commit: db93cba

Metrics [docs]

✅ unchanged

History

💔 Build #263739 failed 16f8f66

kibanamachine · 2025-01-03T16:06:18Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12600439782

## Summary Unskip test that was previously failing in MKI. These tests have been run against MKI. Background on the test -- The test confirms that synthetics integration policies auto update when the package version is updated. It attempts to install an older version of the package and then run fleet setup to install a newer version of the package and trigger auto upgrade Why it failed -- The test expected a lower version of the package to be installed, but a more recent version was already installed. Therefore, when we went to create synthetics integration policies they were installed with the most recent version. How we fixed it -- Before this test, we now uninstall the Synthetics integration to ensure we can force install an earlier version. (cherry picked from commit b851db3)

kibanamachine · 2025-01-03T16:14:45Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…205516) # Backport This will backport the following commits from `main` to `8.x`: - [[Synthetics] chore - synthetics unskip test in mki (#204273)](#204273)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Dominique Clarke <[email protected]>

## Summary Unskips flaky tests. Passed in stateful and serverless for 50 iterations. Previous failures related to fleet integration package failures. The logic for installing the synthetics integration via fleet was changed to improve flakiness in #204273. Those changes have also impacted the flakiness of these tests.

## Summary Unskip test that was previously failing in MKI. These tests have been run against MKI. Background on the test -- The test confirms that synthetics integration policies auto update when the package version is updated. It attempts to install an older version of the package and then run fleet setup to install a newer version of the package and trigger auto upgrade Why it failed -- The test expected a lower version of the package to be installed, but a more recent version was already installed. Therefore, when we went to create synthetics integration policies they were installed with the most recent version. How we fixed it -- Before this test, we now uninstall the Synthetics integration to ensure we can force install an earlier version.

## Summary Unskips flaky tests. Passed in stateful and serverless for 50 iterations. Previous failures related to fleet integration package failures. The logic for installing the synthetics integration via fleet was changed to improve flakiness in elastic#204273. Those changes have also impacted the flakiness of these tests.

## Summary Unskip test that was previously failing in MKI. These tests have been run against MKI. Background on the test -- The test confirms that synthetics integration policies auto update when the package version is updated. It attempts to install an older version of the package and then run fleet setup to install a newer version of the package and trigger auto upgrade Why it failed -- The test expected a lower version of the package to be installed, but a more recent version was already installed. Therefore, when we went to create synthetics integration policies they were installed with the most recent version. How we fixed it -- Before this test, we now uninstall the Synthetics integration to ensure we can force install an earlier version.

## Summary Unskips flaky tests. Passed in stateful and serverless for 50 iterations. Previous failures related to fleet integration package failures. The logic for installing the synthetics integration via fleet was changed to improve flakiness in elastic#204273. Those changes have also impacted the flakiness of these tests.

## Summary Unskip test that was previously failing in MKI. These tests have been run against MKI. Background on the test -- The test confirms that synthetics integration policies auto update when the package version is updated. It attempts to install an older version of the package and then run fleet setup to install a newer version of the package and trigger auto upgrade Why it failed -- The test expected a lower version of the package to be installed, but a more recent version was already installed. Therefore, when we went to create synthetics integration policies they were installed with the most recent version. How we fixed it -- Before this test, we now uninstall the Synthetics integration to ensure we can force install an earlier version.

## Summary Unskips flaky tests. Passed in stateful and serverless for 50 iterations. Previous failures related to fleet integration package failures. The logic for installing the synthetics integration via fleet was changed to improve flakiness in elastic#204273. Those changes have also impacted the flakiness of these tests.

dominiqueclarke added 3 commits December 13, 2024 13:57

synthetics - adjust test expectation to fight flakiness

da29423

Merge branch 'main' of https://github.com/elastic/kibana into chore/s…

b5c3bf2

…ynthetics-unskip-test-in-mki

remove fails on mki tag

4ccfe69

dominiqueclarke added v9.0.0 backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) chore Team:obs-ux-management Observability Management User Experience Team labels Dec 16, 2024

dominiqueclarke marked this pull request as ready for review December 16, 2024 15:19

dominiqueclarke requested review from a team as code owners December 16, 2024 15:19

dominiqueclarke commented Dec 16, 2024

View reviewed changes

dominiqueclarke changed the title ~~Chore/synthetics unskip test in mki~~ [Synthetics] chore - synthetics unskip test in mki Dec 16, 2024

dominiqueclarke added the release_note:skip Skip the PR/issue when compiling release notes label Dec 16, 2024

Merge branch 'main' into chore/synthetics-unskip-test-in-mki

35a7918

benakansara approved these changes Dec 16, 2024

View reviewed changes

wayneseymour requested changes Dec 18, 2024

View reviewed changes

dominiqueclarke added 2 commits January 2, 2025 09:43

Merge branch 'main' into chore/synthetics-unskip-test-in-mki

da6e158

adjust variable naming

16f8f66

Merge branch 'main' into chore/synthetics-unskip-test-in-mki

db93cba

wayneseymour approved these changes Jan 3, 2025

View reviewed changes

dominiqueclarke merged commit b851db3 into elastic:main Jan 3, 2025
8 checks passed

dominiqueclarke deleted the chore/synthetics-unskip-test-in-mki branch January 3, 2025 16:06

kibanamachine mentioned this pull request Jan 3, 2025

[8.x] [Synthetics] chore - synthetics unskip test in mki (#204273) #205516

Merged

kibanamachine added the v8.18.0 label Jan 3, 2025

dominiqueclarke mentioned this pull request Jan 6, 2025

[Synthetics] unskip previously flaky test #205573

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Synthetics] chore - synthetics unskip test in mki #204273

[Synthetics] chore - synthetics unskip test in mki #204273

dominiqueclarke commented Dec 13, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Dec 16, 2024

dominiqueclarke Dec 16, 2024

maryam-saeidi commented Dec 16, 2024

dominiqueclarke commented Dec 16, 2024

maryam-saeidi commented Dec 17, 2024

wayneseymour commented Dec 18, 2024

wayneseymour left a comment

wayneseymour Dec 18, 2024

kibanamachine commented Jan 2, 2025

elasticmachine commented Jan 2, 2025

kibanamachine commented Jan 3, 2025

kibanamachine commented Jan 3, 2025

	const supertestAPI = getService('supertestWithoutAuth');
	const supertestWithoutAuth = getService('supertestWithoutAuth');

[Synthetics] chore - synthetics unskip test in mki #204273

[Synthetics] chore - synthetics unskip test in mki #204273

Conversation

dominiqueclarke commented Dec 13, 2024 • edited by kibanamachine Loading

Summary

Background on the test

Why it failed

How we fixed it

elasticmachine commented Dec 16, 2024

dominiqueclarke Dec 16, 2024

Choose a reason for hiding this comment

maryam-saeidi commented Dec 16, 2024

dominiqueclarke commented Dec 16, 2024

maryam-saeidi commented Dec 17, 2024

wayneseymour commented Dec 18, 2024

wayneseymour left a comment

Choose a reason for hiding this comment

wayneseymour Dec 18, 2024

Choose a reason for hiding this comment

kibanamachine commented Jan 2, 2025

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7639

elasticmachine commented Jan 2, 2025

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Jan 3, 2025

kibanamachine commented Jan 3, 2025

💚 All backports created successfully

Questions ?

dominiqueclarke commented Dec 13, 2024 •

edited by kibanamachine

Loading