Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cloudflare package #984

Merged
merged 19 commits into from
Aug 10, 2021
Merged

Conversation

legoguy1000
Copy link
Contributor

@legoguy1000 legoguy1000 commented May 11, 2021

What does this PR do?

Add Cloudflare integration

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.

How to test this PR locally

cd integrations/packages/cloudflare
elastic-package build && elastic-package stack up -d --services=elasticsearch && $(elastic-package stack shellinit) && elastic-package test pipeline

Related issues

Screenshots

image

image

@elasticmachine
Copy link

elasticmachine commented May 11, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-08-10T07:39:08.343+0000

  • Duration: 12 min 58 sec

  • Commit: 002ccdd

Test stats 🧪

Test Results
Failed 0
Passed 91
Skipped 0
Total 91

Trends 🧪

Image of Build Times

Image of Tests

@P1llus
Copy link
Member

P1llus commented May 26, 2021

run tests

@legoguy1000 legoguy1000 force-pushed the add-cloudflare-package branch from 7c56912 to 9afde7c Compare July 25, 2021 22:29
@legoguy1000 legoguy1000 marked this pull request as ready for review August 1, 2021 20:48
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@legoguy1000 legoguy1000 force-pushed the add-cloudflare-package branch from 5379c47 to 686a8ee Compare August 2, 2021 01:36
Copy link
Contributor

@marc-gr marc-gr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution :D!

Aside from the couple comments, would be nice to add in some of the latest improvements/practices. If you prefer I can push the changes myself or I can help you in doing them:

1- For the ecs fields file, now you can reference directly the ECS definitions from the original repository, take a look at https://github.com/elastic/integrations/pull/971/files#diff-ab0b8ba7ee65eeaec37ad049da3278b9e3f27eafc297ef39b4d97d5557f1e3d7R1 and https://github.com/elastic/integrations/pull/971/files#diff-e25525aa40a63de53545aad0489afe6dd123f5840076dfae30b3f0ad63da1edbR1 for an example, this saves a lot of boilerplate and makes for consistent descriptions across packages.
2- Instead of the _conf field to decide to keep or not the event.original, you can take https://github.com/elastic/integrations/pull/971/files#diff-3e0dc30780e24af54910c65b1217d914992e35cf87045ed76728285b3851ec9bR1 as an example of how we standardised it across all packages, in addition for a custom processors field. You might also want to check how the option is handled in the config and in the pipeline

Also if you can add a couple screenshots of the dashboards that would be great.

Thanks!

@legoguy1000
Copy link
Contributor Author

@marc-gr I think i resolved all the issues you brought up in the first round :).

@legoguy1000
Copy link
Contributor Author

@marc-gr I also found an issue that the geo field groupings are not included as subbordinate fields un the source, destination, client, server, observer as shown here https://github.com/elastic/ecs/blob/v1.10.0/code/go/ecs/destination.go

@marc-gr
Copy link
Contributor

marc-gr commented Aug 9, 2021

@marc-gr I also found an issue that the geo field groupings are not included as subbordinate fields un the source, destination, client, server, observer as shown here https://github.com/elastic/ecs/blob/v1.10.0/code/go/ecs/destination.go

They should be safe to add in those, according to https://www.elastic.co/guide/en/ecs/current/ecs-geo.html, note that AS
is not allowed in observer though https://www.elastic.co/guide/en/ecs/current/ecs-as.html

Copy link
Contributor

@marc-gr marc-gr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! after the last comments I think it should be good to go 👍

@marc-gr marc-gr force-pushed the add-cloudflare-package branch from 52f35f5 to 978a3aa Compare August 9, 2021 13:35
@marc-gr
Copy link
Contributor

marc-gr commented Aug 10, 2021

/test

@marc-gr marc-gr merged commit 3d0f9e2 into elastic:master Aug 10, 2021
eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
* Initial commit

* add dashboards and visualizations

* update dashboards

* update dashboards

* update visualizations

* update

* format package

* update sample log file

* update package

* upodate pipeline

* change datastream to logpull

* update pipeline & vis

* update the dashboards & pipeline

* Updates from comments

* Update fields mappings

* Add httpjson system tests and sample event

* Remove log input and do minor changes to pipeline and configuration

* Update dashboards to use a core saved search

* Export dashboards and add missing event fields

Co-authored-by: Marc Guasch <[email protected]>
@andrewkroh andrewkroh added Integration:cloudflare Cloudflare (Community supported) New Integration Issue or pull request for creating a new integration package. labels Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
7.15 candidate Integration:cloudflare Cloudflare (Community supported) New Integration Issue or pull request for creating a new integration package.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cloudflare
6 participants