Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] For data_stream fields "Filter for value" and "Filter out value" are working opposite to their functionality. #101572

Closed
ghost opened this issue Jun 8, 2021 · 7 comments · Fixed by #101680
Closed

[Security Solution] For data_stream fields "Filter for value" and "Filter out value" are working opposite to their functionality. #101572

ghost opened this issue Jun 8, 2021 · 7 comments · Fixed by #101680
Assignees
@FrankHassanabad
Labels
bug Fixes for quality problems that affect the customer experience fixed impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. QA:Validated Issue has been validated by QA Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0

Comments

@ghost
Copy link

ghost commented Jun 8, 2021

Description
For data_stream fields "Filter for value" and "Filter out value" are working opposite to their functionality.

Build Details:

Version: 7.14.0 SNAPSHOT
Build: 41498
Commit: 5cab87e2069dd31848490f83964291fc802d6889
Artifacts: https://artifacts-api.elastic.co/v1/search/7.14.0-SNAPSHOT

Browser Details:
All

Preconditions:

  1. Elastic stack should be up and running
  2. Endpoint should be install.
  3. Alerts should be generated

Steps to Reproduce:

  1. Navigate to the Detections tab of security
  2. Add the data_stream fields from "Table" tab.
  3. Now use the filters "Filter for value" and observe that no data is displayed.
  4. When use the "filter out value" data is displayed under alerts table.

Impacted Test case:
N/A

Actual Result:
For data_stream fields "Filter for value" and "filter out value" are working opposite to their functionality.

Expected Result:
For data_stream fields "Filter for value" and "filter out value" should be work with their own functionality.

What's working:
This issue is not occurring for event.dataset, event.module
evenet dataset

What's not working:
N/A

Screenshot:
Filter out value
data_stream_filter_out

Filter for value
No_alerts_displayed
@ghost ghost added bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0 labels Jun 8, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@manishgupta-qasource
Copy link

Reviewed & Assigned to @MadameSheema

@manishgupta-qasource manishgupta-qasource added the impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. label Jun 8, 2021
@peluja1012 peluja1012 added the Team:Detections and Resp Security Detection Response Team label Jun 8, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@MadameSheema MadameSheema removed their assignment Jun 8, 2021
@FrankHassanabad FrankHassanabad mentioned this issue Jun 8, 2021
Merged
1 task
@FrankHassanabad FrankHassanabad linked a pull request Jun 8, 2021 that will close this issue
[Security Solution][Detections] Update detection alert mappings to ECS v1.10.0 #101680
Merged
1 task
FrankHassanabad added a commit that referenced this issue Jun 9, 2021
@FrankHassanabad
[Security Solution][Detections] Update detection alert mappings to EC…
1cb17d0 
…S v1.10.0 (#101680)

## Summary

* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.

This should mostly fix:
#101572

Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jun 9, 2021
@FrankHassanabad @kibanamachine
[Security Solution][Detections] Update detection alert mappings to EC…
7ce7a9e 
…S v1.10.0 (elastic#101680)

## Summary

* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.

This should mostly fix:
elastic#101572

Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine added a commit that referenced this issue Jun 10, 2021
@kibanamachine @FrankHassanabad
[Security Solution][Detections] Update detection alert mappings to EC…
5f5cd72 
…S v1.10.0 (#101680) (#101847)

## Summary

* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.

This should mostly fix:
#101572

Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Frank Hassanabad <[email protected]>
semd pushed a commit that referenced this issue Jun 10, 2021
@FrankHassanabad @semd
[Security Solution][Detections] Update detection alert mappings to EC…
88a329e 
…S v1.10.0 (#101680)

## Summary

* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.

This should mostly fix:
#101572

Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
@peluja1012
Copy link
Contributor

@deepikakeshav-qasource could you please validate that this is fixed? Thank you!

@MadameSheema MadameSheema reopened this Jun 14, 2021
@MadameSheema
Copy link
Member

@deepikakeshav-qasource most likely is the latest changes to not be available in the snapshot, so please, validate it in master branch, thanks :)

@ghost
Copy link
Author

ghost commented Jun 14, 2021

Hi @MadameSheema and @peluja1012,

We have validated this ticket on 7.14.0 Master branch and observed that issue is Fixed. data_stream fields "Filter for value" and "filter out value" are working correctly.

Build Details:

Version: 8.0.0-SNAPSHOT
Build Hash:9007199254740991
Artifact : https://artifacts-api.elastic.co/v1/search/8.0.0-SNAPSHOT

Screenshot:
filter_for_value_working

filter_out_value_working

We will re-validate this ticket once the 7.14.0 BC build available.

Thanks!!

@MadameSheema
Copy link
Member

Issue validated. Thanks @deepikakeshav-qasource :)

@ghost ghost added the QA:Validated Issue has been validated by QA label Jun 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FrankHassanabad FrankHassanabad

Labels
bug Fixes for quality problems that affect the customer experience fixed impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. QA:Validated Issue has been validated by QA Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0
Projects
None yet
Milestone
No milestone
5 participants
@peluja1012 @FrankHassanabad @elasticmachine @MadameSheema @manishgupta-qasource