Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[master] Security CCS automation POC #99042

Merged
merged 1 commit into from
May 11, 2021
Merged

[master] Security CCS automation POC #99042

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions x-pack/test/stack_functional_integration/apps/ccs/ccs_discover.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export default ({ getService, getPageObjects }) => {
const kibanaServer = getService('kibanaServer');
const queryBar = getService('queryBar');
const filterBar = getService('filterBar');
const supertest = getService('supertest');

before(async () => {
await browser.setWindowSize(1200, 800);
Expand Down Expand Up @@ -97,6 +98,8 @@ export default ({ getService, getPageObjects }) => {
);
await PageObjects.security.logout();
}
// visit app/security so to create .siem-signals-* as side effect
await PageObjects.common.navigateToApp('security', { insertTimestamp: false });
const url = await browser.getCurrentUrl();
log.debug(url);
if (!url.includes('kibana')) {
Expand Down Expand Up @@ -135,6 +138,35 @@ export default ({ getService, getPageObjects }) => {
expect(patternName).to.be('*:makelogs工程-*');
});

it('create local siem signals index pattern', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in this file it is already tested that an index pattern can be added, in order to improve execution time, as part of the data preparation we need for our test, we can probably add the index we need through the API.

Copy link
Contributor Author

@cavokz cavokz May 5, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such API is available since 7.11 but it's not publicly documented yet. See #99372. The API is described in the original implementation PR #83576.

log.debug('Add index pattern: .siem-signals-*');
await supertest
.post('/api/index_patterns/index_pattern')
.set('kbn-xsrf', 'true')
.send({
index_pattern: {
title: '.siem-signals-*',
},
override: true,
})
.expect(200);
});

it('create remote monitoring ES index pattern', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code inside this test and the above is a preparation of the test we want to perform. So I would refactor it in order to reflect that.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually have 2 other ways to automate creating the index pattern without using the API.

  1. The old way we're trying to move away from is using es_archiver, which let's us save and load a whole .kibana index, which includes the index-pattern saved object. There's still lots of this in use in functional tests. And we do still need it for loading data other than the .kibana index.

  2. kbnarchiver is pretty new. It basically just wraps the saved object API. There's a CLI for it. So the basic steps would be;

  • manually create your index pattern, and any other saved objects you need for your test
  • run the CLI to export the saved objects to a .json file
  • add the .json file to your PR and code to load that file in your test (often in the before method)

Here's an example PR switching from using esArchiver to kbnArchiver #98063

log.debug('Add index pattern: data:.monitoring-es-*');
await supertest
.post('/api/index_patterns/index_pattern')
.set('kbn-xsrf', 'true')
.send({
index_pattern: {
title: 'data:.monitoring-es-*',
timeFieldName: 'timestamp',
},
override: true,
})
.expect(200);
});

it('local:makelogs(star) should discover data from the local cluster', async () => {
await PageObjects.common.navigateToApp('discover', { insertTimestamp: false });

Expand Down Expand Up @@ -203,5 +235,36 @@ export default ({ getService, getPageObjects }) => {
expect(hitCount).to.be.lessThan(originalHitCount);
});
});

it('should generate alerts based on remote events', async () => {
log.debug('Add detection rule type:shards on data:.monitoring-es-*');
await supertest
.post('/api/detection_engine/rules')
.set('kbn-xsrf', 'true')
.send({
description: 'This is the description of the rule',
risk_score: 17,
severity: 'low',
interval: '10s',
name: 'CCS_Detection_test',
type: 'query',
from: 'now-1d',
index: ['data:.monitoring-es-*'],
timestamp_override: 'timestamp',
query: 'type:shards',
language: 'kuery',
enabled: true,
})
.expect(200);

log.debug('Check if any alert got to .siem-signals-*');
await PageObjects.common.navigateToApp('discover', { insertTimestamp: false });
await PageObjects.discover.selectIndexPattern('.siem-signals-*');
await retry.tryForTime(40000, async () => {
const hitCount = await PageObjects.discover.getHitCount();
log.debug('### hit count = ' + hitCount);
expect(hitCount).to.be.greaterThan('0');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better if we assert the number of alerts we are expecting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This requires tuning, depends on the observation period and on the time elapsed when the reading is done. I also suspect that the observed index (.monitoring-es-*) get updated. Essentially, I'm not sure we can have a deterministic value here. Maybe I should investigate some other index for applying the rule onto.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is complex to use our own index of data in the CCS configuration?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As anticipated, none of the indices on the remote (except makelogs*, which is non ECS-compatible) has a deterministic number of entries that can be checked here.

Should we consider ingesting some other data then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, we should.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably a good place to use esArchiver, but it's going to take a little bit of work.

The integration-test repo no longer uses node.js which would be required to call esArchiver.
After the CCS config is provisioned we basically do pushd ../kibana, and then start the FTR tests.

FTR tests do load data with esArchiver, but currently only to the local cluster. While it's possible to add other configuration for FTR so that esArchiver could also load data in a remote cluster, I think it would be easier to use the CLI to call it from the jenkins_test.sh script in the integration-test just before starting the FTR.
If you want to go this route, let me know and I'll help with it.

});
});
});
};