[Security Solution][Endpoint] Host Isolation API changes #113621

ashokaditya · 2021-10-01T13:28:27Z

Summary

Host isolation API changes that take into account if the new endpoint data streams are present.
If yes then add an action request doc to the new data stream as the user before writing to the .fleet-actions index as Kibana.
If no then the host isolation API works as usual.

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

and then the fleet index. Else do as usual. fixes elastic/security-team/issues/1704

elasticmachine · 2021-10-05T14:52:04Z

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts

academo · 2021-10-06T07:36:22Z

I am not fully familiar with the way the indices work. I only had nitpick comments about improving the code. Better if someone else that understand that logic also approves it.

review changes refs elastic/security-team/issues/1704

ashokaditya · 2021-10-11T07:24:42Z

@elasticmachine merge upstream

pzl

minor revisions

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.test.ts

pzl · 2021-10-11T15:47:33Z

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.test.ts

+          { index: string; body: EndpointAction }
+        ] = [
+          (ctx.core.elasticsearch.client.asCurrentUser.index as jest.Mock).mock.calls[0][0],
+          (ctx.core.elasticsearch.client.asInternalUser.index as jest.Mock).mock.calls[1][0],


So this is getting the first argument of the second call to asInternalUser.index(). What's the first thing we index as internal user via this path? Are we guaranteeing that it will always happen in that order?

The first thing that is indexed is .logs-endpoint.actions and then .fleet-actions. This is because of the way I mocked the client to use internalUser when the index exists.

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts

…et-actions` review comments

ashokaditya · 2021-10-12T15:05:18Z

x-pack/plugins/security_solution/common/endpoint/types/actions.ts

-  code: string;
-  id: string;
+  code?: string;
+  id?: string;
  message: string;
-  stack_trace: string;
-  type: string;
+  stack_trace?: string;
+  type?: string;


would be nice to know if some of these can be more specific! ECS doesn't tell much more than some of them being keywords. Can we decide on some error codes for the failed action request errors that we discussed? @pzl

https://www.elastic.co/guide/en/ecs/current/ecs-error.html#field-error-code

ashokaditya · 2021-10-12T15:06:18Z

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts

+      body: {
+        ...doc,
+        error: {
+          code: '424',


This translates to a Failed Dependency http error code. Using a code would be nice to filter out these records quickly

What do you think @pzl

Do the error codes in these responses correlate to HTTP error elsewhere? If we don't have that established, 424 won't mean much the next time we see it until we look it up

Well, not in Kibana that I know of. Should I add a comment here about the specific code? Or should I remove it altogether?

doesn't have a big impact, just leave it

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts

review comment

ashokaditya · 2021-10-12T16:46:08Z

@elasticmachine merge upstream

ashokaditya · 2021-10-12T18:24:56Z

@elasticmachine merge upstream

ashokaditya · 2021-10-13T05:14:39Z

@elasticmachine merge upstream

kibanamachine · 2021-10-13T07:18:12Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💚 Build #160160 succeeded 1bdbe33
💔 Build #160114 failed dddf9b6
💔 Build #160086 failed ad0339d
💔 Build #160039 failed 2890fb6
💚 Build #159576 succeeded 25b6a9d

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ashokaditya

) * Use the new data stream (if exists) to write action request to and then the fleet index. Else do as usual. fixes elastic/security-team/issues/1704 * fix legacy tests * add relevant additional tests * remove duplicate test * update tests * cleanup review changes refs elastic/security-team/issues/1704 * fix lint * Use correct mapping keys when writing to index * write record on new index when action request fails to write to `.fleet-actions` review comments * better error message review comment Co-authored-by: Kibana Machine <[email protected]>

kibanamachine · 2021-10-13T07:28:39Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

…114757) * Use the new data stream (if exists) to write action request to and then the fleet index. Else do as usual. fixes elastic/security-team/issues/1704 * fix legacy tests * add relevant additional tests * remove duplicate test * update tests * cleanup review changes refs elastic/security-team/issues/1704 * fix lint * Use correct mapping keys when writing to index * write record on new index when action request fails to write to `.fleet-actions` review comments * better error message review comment Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Ashokaditya <[email protected]>

…ide-users-to-saving-ux * 'master' of github.com:elastic/kibana: (133 commits) [DOCS] Indicate reports are a subscription feature (elastic#114653) Update namespace for indices (elastic#114612) [DOCS] Adds Logstash pipeline settings (elastic#114648) Bump EPR snapshot version used for tests (elastic#114529) [Security Solution] [Endpoint] Fleet summary card adjustments (elastic#114291) skip flaky suite (elastic#68400) [Visualizations] fix usage of optional dependencies (elastic#114286) [Security Solution] [Detections] Improves custom query rule upgrade test (elastic#114454) [fleet] Add Integration Preference selector (elastic#114432) [Reporting] Add new `data-render-error` attribute (elastic#114472) Replace EuiCodeEditor with CodeEditor in app-services code (elastic#114316) [data views] add getDefaultDataView method (elastic#113891) [Security Solution] [Endpoint] Event filters uses the new card design (elastic#114126) [fleet] Tweak Header UI (elastic#114704) [APM] Filter on tx metrics for instance stats (elastic#114758) [APM] Fix typo in linting docs (elastic#114764) [Discover] Removing SavedObject usage for savedSearch (elastic#112983) [Fleet] Add Integration Policy Page Improvements (elastic#114556) [Lens] Keep the custom label when transitioning to/from Formula (elastic#114270) [Security Solution][Endpoint] Host Isolation API changes (elastic#113621) ...

* rename legacy actions/responses fixes elastic/security-team/issues/1702 * use correct name for responses index refs /pull/113621 * extract helper method to utils * append endpoint responses docs to activity log * Show completed responses on activity log fixes elastic/security-team/issues/1703 * remove width restriction on date picker * add a simple test to verify endpoint responses fixes elastic/security-team/issues/1702 * find unique action_ids from `.fleet-actions` and `.logs-endpoint.actions-default` indices fixes elastic/security-team/issues/1702 * do not filter out endpoint only actions/responses that did not make it to Fleet review comments * use a constant to manage various doc types review comments * refactor `getActivityLog` Simplify `getActivityLog` so it is easier to reason with. review comments * skip this for now will mock this better in a new PR * improve types * display endpoint actions similar to fleet actions, but with success icon color * Correctly do mocks for tests * Include only errored endpoint actions, remove successful duplicates fixes elastic/security-team/issues/1703 * Update tests to use non duplicate action_ids review comments fixes elastic/security-team/issues/1703 * show correct action title review fixes * statusCode constant review change * rename review changes * Update translations.ts refs 74a8340 Co-authored-by: Kibana Machine <[email protected]>

) * rename legacy actions/responses fixes elastic/security-team/issues/1702 * use correct name for responses index refs elastic/pull/113621 * extract helper method to utils * append endpoint responses docs to activity log * Show completed responses on activity log fixes elastic/security-team/issues/1703 * remove width restriction on date picker * add a simple test to verify endpoint responses fixes elastic/security-team/issues/1702 * find unique action_ids from `.fleet-actions` and `.logs-endpoint.actions-default` indices fixes elastic/security-team/issues/1702 * do not filter out endpoint only actions/responses that did not make it to Fleet review comments * use a constant to manage various doc types review comments * refactor `getActivityLog` Simplify `getActivityLog` so it is easier to reason with. review comments * skip this for now will mock this better in a new PR * improve types * display endpoint actions similar to fleet actions, but with success icon color * Correctly do mocks for tests * Include only errored endpoint actions, remove successful duplicates fixes elastic/security-team/issues/1703 * Update tests to use non duplicate action_ids review comments fixes elastic/security-team/issues/1703 * show correct action title review fixes * statusCode constant review change * rename review changes * Update translations.ts refs elastic@74a8340 Co-authored-by: Kibana Machine <[email protected]>

…115492) * rename legacy actions/responses fixes elastic/security-team/issues/1702 * use correct name for responses index refs /pull/113621 * extract helper method to utils * append endpoint responses docs to activity log * Show completed responses on activity log fixes elastic/security-team/issues/1703 * remove width restriction on date picker * add a simple test to verify endpoint responses fixes elastic/security-team/issues/1702 * find unique action_ids from `.fleet-actions` and `.logs-endpoint.actions-default` indices fixes elastic/security-team/issues/1702 * do not filter out endpoint only actions/responses that did not make it to Fleet review comments * use a constant to manage various doc types review comments * refactor `getActivityLog` Simplify `getActivityLog` so it is easier to reason with. review comments * skip this for now will mock this better in a new PR * improve types * display endpoint actions similar to fleet actions, but with success icon color * Correctly do mocks for tests * Include only errored endpoint actions, remove successful duplicates fixes elastic/security-team/issues/1703 * Update tests to use non duplicate action_ids review comments fixes elastic/security-team/issues/1703 * show correct action title review fixes * statusCode constant review change * rename review changes * Update translations.ts refs 74a8340 Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Ashokaditya <[email protected]>

ashokaditya self-assigned this Oct 1, 2021

ashokaditya added auto-backport Deprecated - use backport:version if exact versions are needed release_note:breaking release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v7.16.0 v8.0.0 labels Oct 1, 2021

ashokaditya added 2 commits October 5, 2021 09:17

Use the new data stream (if exists) to write action request to

2c3817d

and then the fleet index. Else do as usual. fixes elastic/security-team/issues/1704

fix legacy tests

d9a4961

ashokaditya marked this pull request as ready for review October 5, 2021 14:52

ashokaditya requested a review from a team as a code owner October 5, 2021 14:52

ashokaditya requested review from academo and dasansol92 October 5, 2021 14:52

ashokaditya requested a review from pzl October 5, 2021 14:52

ashokaditya added 2 commits October 5, 2021 17:02

add relevant additional tests

7334401

remove duplicate test

1dda776

academo reviewed Oct 6, 2021

View reviewed changes

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts Outdated Show resolved Hide resolved

academo reviewed Oct 6, 2021

View reviewed changes

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts Outdated Show resolved Hide resolved

academo reviewed Oct 6, 2021

View reviewed changes

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts Outdated Show resolved Hide resolved

academo reviewed Oct 6, 2021

View reviewed changes

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts Outdated Show resolved Hide resolved

academo reviewed Oct 6, 2021

View reviewed changes

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts Outdated Show resolved Hide resolved

academo reviewed Oct 6, 2021

View reviewed changes

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts Outdated Show resolved Hide resolved

academo approved these changes Oct 6, 2021

View reviewed changes

academo requested review from academo and removed request for academo October 6, 2021 07:36

ashokaditya added 3 commits October 6, 2021 12:05

update tests

1902317

cleanup

e1e3d4e

review changes refs elastic/security-team/issues/1704

fix lint

64fcd1f

kibanamachine and others added 2 commits October 11, 2021 03:24

Merge branch 'master' into feat/olm-host_isolation_api_changes-1704

07d37aa

Use correct mapping keys when writing to index

25b6a9d

pzl reviewed Oct 11, 2021

View reviewed changes

write record on new index when action request fails to write to `.fle…

2890fb6

…et-actions` review comments

ashokaditya commented Oct 12, 2021

View reviewed changes

pzl reviewed Oct 12, 2021

View reviewed changes

x-pack/plugins/security_solution/server/endpoint/routes/actions/isolation.ts Outdated Show resolved Hide resolved

better error message

ad0339d

review comment

Merge branch 'master' into feat/olm-host_isolation_api_changes-1704

dddf9b6

Merge branch 'master' into feat/olm-host_isolation_api_changes-1704

1bdbe33

pzl approved these changes Oct 12, 2021

View reviewed changes

Merge branch 'master' into feat/olm-host_isolation_api_changes-1704

ef42610

ashokaditya removed the release_note:breaking label Oct 13, 2021

ashokaditya merged commit 1d71d42 into elastic:master Oct 13, 2021

ashokaditya deleted the feat/olm-host_isolation_api_changes-1704 branch October 13, 2021 07:25

kibanamachine mentioned this pull request Oct 13, 2021

[7.x] [Security Solution][Endpoint] Host Isolation API changes (#113621) #114757

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][Endpoint] Host Isolation API changes #113621

[Security Solution][Endpoint] Host Isolation API changes #113621

ashokaditya commented Oct 1, 2021 •

edited

Loading

elasticmachine commented Oct 5, 2021

academo commented Oct 6, 2021

ashokaditya commented Oct 11, 2021

pzl left a comment

pzl Oct 11, 2021

ashokaditya Oct 12, 2021 •

edited

Loading

ashokaditya Oct 12, 2021

ashokaditya Oct 12, 2021

ashokaditya Oct 12, 2021

pzl Oct 12, 2021

ashokaditya Oct 12, 2021

pzl Oct 12, 2021

ashokaditya commented Oct 12, 2021

ashokaditya commented Oct 12, 2021

ashokaditya commented Oct 13, 2021

kibanamachine commented Oct 13, 2021

kibanamachine commented Oct 13, 2021

[Security Solution][Endpoint] Host Isolation API changes #113621

[Security Solution][Endpoint] Host Isolation API changes #113621

Conversation

ashokaditya commented Oct 1, 2021 • edited Loading

Summary

Checklist

elasticmachine commented Oct 5, 2021

academo commented Oct 6, 2021

ashokaditya commented Oct 11, 2021

pzl left a comment

Choose a reason for hiding this comment

pzl Oct 11, 2021

Choose a reason for hiding this comment

ashokaditya Oct 12, 2021 • edited Loading

Choose a reason for hiding this comment

ashokaditya Oct 12, 2021

Choose a reason for hiding this comment

ashokaditya Oct 12, 2021

Choose a reason for hiding this comment

ashokaditya Oct 12, 2021

Choose a reason for hiding this comment

pzl Oct 12, 2021

Choose a reason for hiding this comment

ashokaditya Oct 12, 2021

Choose a reason for hiding this comment

pzl Oct 12, 2021

Choose a reason for hiding this comment

ashokaditya commented Oct 12, 2021

ashokaditya commented Oct 12, 2021

ashokaditya commented Oct 13, 2021

kibanamachine commented Oct 13, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Oct 13, 2021

💚 Backport successful

ashokaditya commented Oct 1, 2021 •

edited

Loading

ashokaditya Oct 12, 2021 •

edited

Loading