Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: X-Pack Fleet Agent Policy API Integration Tests.x-pack/test/fleet_api_integration/apis/agent_policy/agent_policy·ts - Agent policies fleet_agent_policies POST /api/fleet/agent_policies should create .fleet-policies document with inputs #203346

Closed
kibanamachine opened this issue Dec 6, 2024 · 20 comments
Closed

Failing test: X-Pack Fleet Agent Policy API Integration Tests.x-pack/test/fleet_api_integration/apis/agent_policy/agent_policy·ts - Agent policies fleet_agent_policies POST /api/fleet/agent_policies should create .fleet-policies document with inputs #203346

kibanamachine opened this issue Dec 6, 2024 · 20 comments
Assignees
@jen-huang
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Fleet Team label for Observability Data Collection Fleet team

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Dec 6, 2024 •
edited
Loading

A test failed on a tracked branch

Error: expected 4 to sort of equal 3
    at Assertion.assert (expect.js:100:11)
    at Assertion.eql (expect.js:244:8)
    at Context.<anonymous> (agent_policy.ts:321:48)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:74:16) {
  actual: '4',
  expected: '3',
  showDiff: true
}

First failure: kibana-on-merge - 8.x
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Dec 6, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Dec 6, 2024
@kibanamachine kibanamachine added the Team:Fleet Team label for Observability Data Collection Fleet team label Dec 6, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Dec 6, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.17

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.17

mistic added a commit that referenced this issue Dec 7, 2024
@mistic
skip flaky suite (#203346)
68a94b7
mistic added a commit to mistic/kibana that referenced this issue Dec 7, 2024
@mistic
skip flaky suite (elastic#203346)
6ffddd9
mistic added a commit that referenced this issue Dec 7, 2024
@mistic
skip flaky suite (#203346)
04111eb
@mistic
Copy link
Member

mistic commented Dec 7, 2024 •
edited
Loading

Skipped.

main: 6ffddd9
8.x: 68a94b7
8.17: 04111eb

@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-snapshot-verify - 8.17

@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-snapshot-verify - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-snapshot-verify - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-snapshot-verify - 8.17

@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-snapshot-verify - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-snapshot-verify - main

markov00 pushed a commit to markov00/kibana that referenced this issue Dec 7, 2024
@mistic @markov00
skip flaky suite (elastic#203346)
c336db1
SoniaSanzV pushed a commit to SoniaSanzV/kibana that referenced this issue Dec 9, 2024
@mistic @SoniaSanzV
skip flaky suite (elastic#203346)
076f049
SoniaSanzV pushed a commit to SoniaSanzV/kibana that referenced this issue Dec 9, 2024
@mistic @SoniaSanzV
skip flaky suite (elastic#203346)
b05ee66
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 9, 2024
@mistic @CAWilson94
skip flaky suite (elastic#203346)
f11228f
Samiul-TheSoccerFan pushed a commit to Samiul-TheSoccerFan/kibana that referenced this issue Dec 10, 2024
@mistic @Samiul-TheSoccerFan
skip flaky suite (elastic#203346)
88a7d06
@kpollich kpollich self-assigned this Dec 10, 2024
mykolaharmash pushed a commit to mykolaharmash/kibana that referenced this issue Dec 11, 2024
@mistic @mykolaharmash
skip flaky suite (elastic#203346)
9eeb8b1
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@mistic @CAWilson94
skip flaky suite (elastic#203346)
4733beb
@kpollich kpollich removed their assignment Jan 9, 2025
@jen-huang jen-huang self-assigned this Jan 21, 2025
@jen-huang jen-huang mentioned this issue Jan 21, 2025
Merged
3 tasks
jen-huang added a commit that referenced this issue Jan 24, 2025
@jen-huang @kibanamachine
[UII] Restrict non-local ES output types for agentless integrations a…
d4cc532 
…nd policies (#207296)

## Summary

Resolves #202090.
Incidentally also fixes and unskips the failing test reported in
#203346.

This PR makes it so that:

- Agentless agent policies and package policies (those with
`supports_agentless: true` set) cannot be directly set to use a
non-local ES output for integration data
  - This restriction applies to UI and API level
- When a non-local ES output would be updated to be the global Fleet
default integration data output, existing agentless policies without an
explicit output set will have their output directly set to the current
default output ID
- This is the same mechanism used today to ensure that Fleet Server and
Synthetics integrations do not accidentally have their output set to
non-ES as well

## Testing
1. Apply
[patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4)
to skip actual agentless creation
2. Create different types of outputs in addition to default local ES
3. Enable beta integrations
4. Add an agentless integration, for example Box Connector, observe that
outputs list only shows ES outputs:
<img width="1422" alt="image"
src="https://github.com/user-attachments/assets/72e43220-702f-4bb7-8e37-8be69aa4e6ea"
/>
5. Switch to agent-based setup technology, observe that outputs list now
shows all outputs
6. Create the agentless integration, go to its agent policy
7. Observe that outputs list only enables ES outputs:
<img width="1425" alt="image"
src="https://github.com/user-attachments/assets/3bc5985f-07bf-407a-8b62-4248b28904a5"
/>
8. Play around with setting global default outputs, it should be not
possible to get into a state where an agentless policy is using a
non-local ES output

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <[email protected]>
@jen-huang
Copy link
Contributor

Fixed and re-enabled in #207296

jen-huang added a commit to jen-huang/kibana that referenced this issue Jan 24, 2025
@jen-huang
[UII] Restrict non-local ES output types for agentless integrations a…
f961e15 
…nd policies (elastic#207296)

## Summary

Resolves elastic#202090.
Incidentally also fixes and unskips the failing test reported in
elastic#203346.

This PR makes it so that:

- Agentless agent policies and package policies (those with
`supports_agentless: true` set) cannot be directly set to use a
non-local ES output for integration data
  - This restriction applies to UI and API level
- When a non-local ES output would be updated to be the global Fleet
default integration data output, existing agentless policies without an
explicit output set will have their output directly set to the current
default output ID
- This is the same mechanism used today to ensure that Fleet Server and
Synthetics integrations do not accidentally have their output set to
non-ES as well

## Testing
1. Apply
[patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4)
to skip actual agentless creation
2. Create different types of outputs in addition to default local ES
3. Enable beta integrations
4. Add an agentless integration, for example Box Connector, observe that
outputs list only shows ES outputs:
<img width="1422" alt="image"
src="https://github.com/user-attachments/assets/72e43220-702f-4bb7-8e37-8be69aa4e6ea"
/>
5. Switch to agent-based setup technology, observe that outputs list now
shows all outputs
6. Create the agentless integration, go to its agent policy
7. Observe that outputs list only enables ES outputs:
<img width="1425" alt="image"
src="https://github.com/user-attachments/assets/3bc5985f-07bf-407a-8b62-4248b28904a5"
/>
8. Play around with setting global default outputs, it should be not
possible to get into a state where an agentless policy is using a
non-local ES output

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit d4cc532)

# Conflicts:
#	oas_docs/bundle.json
#	oas_docs/bundle.serverless.json
#	oas_docs/output/kibana.yaml
#	x-pack/platform/plugins/shared/fleet/server/services/agent_policy.ts
#	x-pack/platform/plugins/shared/fleet/server/services/output.test.ts
#	x-pack/platform/plugins/shared/fleet/server/types/models/agent_policy.ts
#	x-pack/test/fleet_api_integration/apis/agent_policy/agent_policy.ts
JoseLuisGJ pushed a commit to JoseLuisGJ/kibana that referenced this issue Jan 27, 2025
@jen-huang @kibanamachine @JoseLuisGJ
[UII] Restrict non-local ES output types for agentless integrations a…
2d0defd 
…nd policies (elastic#207296)

## Summary

Resolves elastic#202090.
Incidentally also fixes and unskips the failing test reported in
elastic#203346.

This PR makes it so that:

- Agentless agent policies and package policies (those with
`supports_agentless: true` set) cannot be directly set to use a
non-local ES output for integration data
  - This restriction applies to UI and API level
- When a non-local ES output would be updated to be the global Fleet
default integration data output, existing agentless policies without an
explicit output set will have their output directly set to the current
default output ID
- This is the same mechanism used today to ensure that Fleet Server and
Synthetics integrations do not accidentally have their output set to
non-ES as well

## Testing
1. Apply
[patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4)
to skip actual agentless creation
2. Create different types of outputs in addition to default local ES
3. Enable beta integrations
4. Add an agentless integration, for example Box Connector, observe that
outputs list only shows ES outputs:
<img width="1422" alt="image"
src="https://github.com/user-attachments/assets/72e43220-702f-4bb7-8e37-8be69aa4e6ea"
/>
5. Switch to agent-based setup technology, observe that outputs list now
shows all outputs
6. Create the agentless integration, go to its agent policy
7. Observe that outputs list only enables ES outputs:
<img width="1425" alt="image"
src="https://github.com/user-attachments/assets/3bc5985f-07bf-407a-8b62-4248b28904a5"
/>
8. Play around with setting global default outputs, it should be not
possible to get into a state where an agentless policy is using a
non-local ES output

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jen-huang jen-huang

Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Fleet Team label for Observability Data Collection Fleet team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jen-huang @mistic @kpollich @elasticmachine @kibanamachine