Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/awsfirehosereceiver] Support logs from AWS services sent directly to Firehose #36379

Open
kaiyan-sheng opened this issue Nov 14, 2024 · 2 comments
Labels
enhancement New feature or request needs triage New item requiring triage receiver/awsfirehose Stale

Comments

@kaiyan-sheng
Copy link
Contributor

Component(s)

receiver/awsfirehose

Is your feature request related to a problem? Please describe.

The current AWS Firehose receiver only supports cwlogs record type which is ingesting JSON format logs from CloudWatch. We should also add support for logs sent from services directly to Firehose. For example VPC flow logs can be sent directly to Firehose and our awsfirehosereceiver should be able to process these requests as well.

384160617-bcb3efed-6f4a-45bc-8730-4b96252a0b5d

Describe the solution you'd like

We can introduce a new record_type: firehoselogs to separate the log type from record_type: cwlogs. Since there can be logs from different AWS services send directly to Firehose, the format will be different too. This record_type: firehoselogs will process the request, store the whole record data into message field, store the timestamp and firehose ARN.

Describe alternatives you've considered

No response

Additional context

No response

@kaiyan-sheng kaiyan-sheng added enhancement New feature or request needs triage New item requiring triage labels Nov 14, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request needs triage New item requiring triage receiver/awsfirehose Stale
Projects
None yet
Development

No branches or pull requests

1 participant