Adjust aws package to use input groups (#767)

elastic · Jun 29, 2021 · af9597a · af9597a
1 parent bf483ce
commit af9597a
Show file tree

Hide file tree

Showing 60 changed files with 3,530 additions and 3,225 deletions.
diff --git a/packages/aws/_dev/build/docs/README.md b/packages/aws/_dev/build/docs/README.md
@@ -95,164 +95,3 @@ In order to enable AWS integration, please make sure these permissions are given
 * sqs:ListQueues
 * sts:GetCallerIdentity
 * iam:ListAccountAliases
-
-## Logs
-
-### cloudtrail
-
-The `cloudtrail` dataset collects the AWS CloudTrail logs. CloudTrail monitors 
-events for the account. If user creates a trail, it delivers those events as log
- files to a specific Amazon S3 bucket. The `cloudtrail` dataset does not read 
- the CloudTrail Digest files that are delivered to the S3 bucket when Log File 
- Integrity is turned on, it only reads the CloudTrail logs.
-
-{{fields "cloudtrail"}}
-
-### cloudwatch
-
-The `cloudwatch` dataset collects CloudWatch logs. Users can use Amazon 
-CloudWatch logs to monitor, store, and access log files from different sources. 
-Export logs from log groups to an Amazon S3 bucket which has SQS notification 
-setup already.
-
-{{fields "cloudwatch_logs"}}
-
-### ec2
-
-The `ec2` dataset is specifically for EC2 logs stored in AWS CloudWatch. Export logs
-from log groups to Amazon S3 bucket which has SQS notification setup already.
-With this dataset, EC2 logs will be parsed into fields like  `ip_address`
-and `process.name`. For logs from other services, please use `cloudwatch` dataset.
-
-{{fields "ec2_logs"}}
-
-### elb
-
-The `elb` dataset collects logs from AWS ELBs. Elastic Load Balancing provides 
-access logs that capture detailed information about requests sent to the load 
-balancer. Each log contains information such as the time the request was 
-received, the client's IP address, latencies, request paths, and server 
-responses. Users can use these access logs to analyze traffic patterns and to 
-troubleshoot issues.
-
-Please follow [enable access logs for classic load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html)
-for sending Classic ELB access logs to S3 bucket.
-For application load balancer, please follow [enable access log for application load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#enable-access-logging).
-For network load balancer, please follow [enable access log for network load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest//network/load-balancer-access-logs.html).
-
-{{fields "elb_logs"}}
-
-### s3access
-
-The `s3access` dataset collects server access logs from AWS S3. Server access 
-logging provides detailed records for the requests that are made to a bucket. 
-Server access logs are useful for many applications. For example, access log 
-information can be useful in security and access audits. It can also help users
-to learn about customer base and understand Amazon S3 bill.
-
-Please follow [how to enable server access logging](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html#server-access-logging-overview)
-for sending server access logs to S3 bucket.
-
-{{fields "s3access"}}
-
-### vpcflow
-
-{{fields "vpcflow"}}
-
-## Metrics
-
-### billing
-
-{{event "billing"}}
-
-{{fields "billing"}}
-
-### cloudwatch
-
-{{event "cloudwatch_metrics"}}
-
-{{fields "cloudwatch_metrics"}}
-
-### dynamodb
-
-{{event "dynamodb"}}
-
-{{fields "dynamodb"}}
-
-### ebs
-
-{{event "ebs"}}
-
-{{fields "ebs"}}
-
-### ec2
-
-{{event "ec2_metrics"}}
-
-{{fields "ec2_metrics"}}
-
-### elb
-
-{{event "elb_metrics"}}
-
-{{fields "elb_metrics"}}
-
-### lambda
-
-{{event "lambda"}}
-
-{{fields "lambda"}}
-
-### natgateway
-
-{{event "natgateway"}}
-
-{{fields "natgateway"}}
-
-### rds
-
-{{event "rds"}}
-
-{{fields "rds"}}
-
-### s3_daily_storage
-
-{{event "s3_daily_storage"}}
-
-{{fields "s3_daily_storage"}}
-
-### s3_request
-
-{{event "s3_request"}}
-
-{{fields "s3_request"}}
-
-### sns
-
-{{event "sns"}}
-
-{{fields "sns"}}
-
-### sqs
-
-{{event "sqs"}}
-
-{{fields "sqs"}}
-
-### transitgateway
-
-{{event "transitgateway"}}
-
-{{fields "transitgateway"}}
-
-### usage
-
-{{event "usage"}}
-
-{{fields "usage"}}
-
-### vpn
-
-{{event "vpn"}}
-
-{{fields "vpn"}}
diff --git a/packages/aws/_dev/build/docs/billing.md b/packages/aws/_dev/build/docs/billing.md
@@ -0,0 +1,7 @@
+# billing
+
+## Metrics
+
+{{event "billing"}}
+
+{{fields "billing"}}
diff --git a/packages/aws/_dev/build/docs/cloudtrail.md b/packages/aws/_dev/build/docs/cloudtrail.md
@@ -0,0 +1,11 @@
+# cloudtrail
+
+## Logs
+
+The `cloudtrail` dataset collects the AWS CloudTrail logs. CloudTrail monitors 
+events for the account. If user creates a trail, it delivers those events as log
+ files to a specific Amazon S3 bucket. The `cloudtrail` dataset does not read 
+ the CloudTrail Digest files that are delivered to the S3 bucket when Log File 
+ Integrity is turned on, it only reads the CloudTrail logs.
+
+{{fields "cloudtrail"}}
diff --git a/packages/aws/_dev/build/docs/cloudwatch.md b/packages/aws/_dev/build/docs/cloudwatch.md
@@ -0,0 +1,16 @@
+# cloudwatch
+
+## Logs
+
+The `cloudwatch` dataset collects CloudWatch logs. Users can use Amazon 
+CloudWatch logs to monitor, store, and access log files from different sources. 
+Export logs from log groups to an Amazon S3 bucket which has SQS notification 
+setup already.
+
+{{fields "cloudwatch_logs"}}
+
+## Metrics
+
+{{event "cloudwatch_metrics"}}
+
+{{fields "cloudwatch_metrics"}}
diff --git a/packages/aws/_dev/build/docs/dynamodb.md b/packages/aws/_dev/build/docs/dynamodb.md
@@ -0,0 +1,7 @@
+# dynamodb
+
+## Metrics
+
+{{event "dynamodb"}}
+
+{{fields "dynamodb"}}
diff --git a/packages/aws/_dev/build/docs/ebs.md b/packages/aws/_dev/build/docs/ebs.md
@@ -0,0 +1,7 @@
+# ebs
+
+## Metrics
+
+{{event "ebs"}}
+
+{{fields "ebs"}}
diff --git a/packages/aws/_dev/build/docs/ec2.md b/packages/aws/_dev/build/docs/ec2.md
@@ -0,0 +1,16 @@
+# ec2
+
+## Logs
+
+The `ec2` dataset is specifically for EC2 logs stored in AWS CloudWatch. Export logs
+from log groups to Amazon S3 bucket which has SQS notification setup already.
+With this dataset, EC2 logs will be parsed into fields like  `ip_address`
+and `process.name`. For logs from other services, please use `cloudwatch` dataset.
+
+{{fields "ec2_logs"}}
+
+## Metrics
+
+{{event "ec2_metrics"}}
+
+{{fields "ec2_metrics"}}
diff --git a/packages/aws/_dev/build/docs/elb.md b/packages/aws/_dev/build/docs/elb.md
@@ -0,0 +1,23 @@
+# elb
+
+## Logs
+
+The `elb` dataset collects logs from AWS ELBs. Elastic Load Balancing provides 
+access logs that capture detailed information about requests sent to the load 
+balancer. Each log contains information such as the time the request was 
+received, the client's IP address, latencies, request paths, and server 
+responses. Users can use these access logs to analyze traffic patterns and to 
+troubleshoot issues.
+
+Please follow [enable access logs for classic load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html)
+for sending Classic ELB access logs to S3 bucket.
+For application load balancer, please follow [enable access log for application load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#enable-access-logging).
+For network load balancer, please follow [enable access log for network load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest//network/load-balancer-access-logs.html).
+
+{{fields "elb_logs"}}
+
+## Metrics
+
+{{event "elb_metrics"}}
+
+{{fields "elb_metrics"}}
diff --git a/packages/aws/_dev/build/docs/lambda.md b/packages/aws/_dev/build/docs/lambda.md
@@ -0,0 +1,7 @@
+# lambda
+
+## Metrics
+
+{{event "lambda"}}
+
+{{fields "lambda"}}
diff --git a/packages/aws/_dev/build/docs/natgateway.md b/packages/aws/_dev/build/docs/natgateway.md
@@ -0,0 +1,7 @@
+# natgateway
+
+## Metrics
+
+{{event "natgateway"}}
+
+{{fields "natgateway"}}
diff --git a/packages/aws/_dev/build/docs/rds.md b/packages/aws/_dev/build/docs/rds.md
@@ -0,0 +1,7 @@
+# rds
+
+## Metrics
+
+{{event "rds"}}
+
+{{fields "rds"}}
diff --git a/packages/aws/_dev/build/docs/s3.md b/packages/aws/_dev/build/docs/s3.md
@@ -0,0 +1,27 @@
+# S3
+
+## Logs
+The `s3access` dataset collects server access logs from AWS S3. Server access 
+logging provides detailed records for the requests that are made to a bucket. 
+Server access logs are useful for many applications. For example, access log 
+information can be useful in security and access audits. It can also help users
+to learn about customer base and understand Amazon S3 bill.
+
+Please follow [how to enable server access logging](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html#server-access-logging-overview)
+for sending server access logs to S3 bucket.
+
+{{fields "s3access"}}
+
+## Metrics
+
+### s3_daily_storage
+
+{{event "s3_daily_storage"}}
+
+{{fields "s3_daily_storage"}}
+
+### s3_request
+
+{{event "s3_request"}}
+
+{{fields "s3_request"}}
diff --git a/packages/aws/_dev/build/docs/sns.md b/packages/aws/_dev/build/docs/sns.md
@@ -0,0 +1,7 @@
+# sns
+
+## Metrics
+
+{{event "sns"}}
+
+{{fields "sns"}}
diff --git a/packages/aws/_dev/build/docs/sqs.md b/packages/aws/_dev/build/docs/sqs.md
@@ -0,0 +1,7 @@
+# sqs
+
+## Metrics
+
+{{event "sqs"}}
+
+{{fields "sqs"}}
diff --git a/packages/aws/_dev/build/docs/transitgateway.md b/packages/aws/_dev/build/docs/transitgateway.md
@@ -0,0 +1,7 @@
+# transitgateway
+
+## Metrics
+
+{{event "transitgateway"}}
+
+{{fields "transitgateway"}}
diff --git a/packages/aws/_dev/build/docs/usage.md b/packages/aws/_dev/build/docs/usage.md
@@ -0,0 +1,7 @@
+# usage
+
+## Metrics
+
+{{event "usage"}}
+
+{{fields "usage"}}
diff --git a/packages/aws/_dev/build/docs/vpcflow.md b/packages/aws/_dev/build/docs/vpcflow.md
@@ -0,0 +1,5 @@
+# vpcflow
+
+## Logs
+
+{{fields "vpcflow"}}
diff --git a/packages/aws/_dev/build/docs/vpn.md b/packages/aws/_dev/build/docs/vpn.md
@@ -0,0 +1,7 @@
+# vpn
+
+## Metrics
+
+{{event "vpn"}}
+
+{{fields "vpn"}}
diff --git a/packages/aws/changelog.yml b/packages/aws/changelog.yml
@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "0.7.0"
+  changes:
+    - description: Introduce granularity using input_groups
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/767
 - version: "0.6.4"
   changes:
     - description: Add support for Splunk authorization tokens