Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue #9931 - [Filebeat] Add Zeek (formerly Bro) Module #10034

Merged
merged 19 commits into from
Jan 18, 2019
Merged

issue #9931 - [Filebeat] Add Zeek (formerly Bro) Module #10034

merged 19 commits into from
Jan 18, 2019

Conversation

alakahakai
Copy link

@alakahakai alakahakai commented Jan 12, 2019

Fixes issue #9931, with support for Zeek/Bro conn.log, dns.log, http.log, files.log, and ssl.log

@alakahakai alakahakai added in progress Pull request is currently in progress. module review Filebeat Filebeat labels Jan 12, 2019
@alakahakai alakahakai requested review from webmat and ruflin January 12, 2019 09:18
@alakahakai alakahakai requested review from a team as code owners January 12, 2019 09:18
@alakahakai alakahakai removed the review label Jan 12, 2019
…ter clean up and redo, things should be fine.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/secops

@urso urso removed the request for review from a team January 12, 2019 16:24
@alakahakai alakahakai requested a review from andrewkroh January 12, 2019 17:30
@alakahakai alakahakai removed the in progress Pull request is currently in progress. label Jan 12, 2019
@andrewkroh andrewkroh added the ecs label Jan 14, 2019

## Caveats

* Module is to be considered _alpha_.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a notion of Alpha in Elastic, we use either Beta or Experimental. Both Beta and Experimental are not recommended for production and our support team accepts Sev-3 issues only. The difference between them is that for Beta we have clear plans to move them to GA, while Experimental features might go away if unsuccessful. In this case, I'd say Beta fits best.

## Caveats

* Module is to be considered _alpha_.
* Field names will be changing for 7.0 to comply with Elastic Common Schema (ECS).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll only include this module in 7.0 anyway.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not pay attention to this sentence. Will remove this.

brew install bro
```

Configure it to process network traffic and generate logs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will look better as a list, i.e. start each line with *.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree. Will update that

* Module is to be considered _alpha_.
* Field names will be changing for 7.0 to comply with Elastic Common Schema (ECS).

## How to try the module from source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great tutorial, thanks for writing it! I think that after we have the module in a released version, we might want to switch to use the downloaded binaries, which should make it a bit easier.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I can rename this to README-developer.md, and introduce a README.md for normal users that use binary distributions.

@alakahakai
Copy link
Author

Is this beats-ci build error common?

09:00:01 testing_test.go:86: Failed to connect: dial tcp 127.0.0.1:43871: i/o timeout
09:00:01 testing_test.go:57: accept tcp 127.0.0.1:43871: use of closed network connection
09:00:01 2019/01/17 17:00:01 [ERR] socks: Failed to handle request: Failed to resolve destination 'invalid.dns.fqdn-unknown.invalid': lookup invalid.dns.fqdn-unknown.invalid on 127.0.0.11:53: no such host
09:00:01 2019/01/17 17:00:01 [ERR] socks: Failed to handle request: Failed to resolve destination 'invalid.dns.fqdn-unknown.invalid': lookup invalid.dns.fqdn-unknown.invalid on 127.0.0.11:53: no such host
09:00:01 2019/01/17 17:00:01 [ERR] socks: Failed to handle request: readfrom tcp 127.0.0.1:57208->127.0.0.1:41629: splice: broken pipe
09:00:01 2019/01/17 17:00:01 [ERR] socks: Failed to handle request: readfrom tcp 127.0.0.1:54560->127.0.0.1:42527: splice: broken pipe
09:00:01 FAIL

@alakahakai alakahakai added the in progress Pull request is currently in progress. label Jan 17, 2019
@alakahakai
Copy link
Author

Jenkins, test this

@ruflin
Copy link
Contributor

ruflin commented Jan 18, 2019

@alakahakai For now ignore the jenkins CI as it is broken currently.

Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done. It will certainly be beneficial for ECS to define fields for TLS and DNS so our different modules and beats can converge on the field names.

Can you shrink down the screenshot a bit? It's kind of large at 1.5MB.

@alakahakai alakahakai merged commit eee0c50 into elastic:master Jan 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ecs Filebeat Filebeat in progress Pull request is currently in progress. module review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants