Add custom headers

[cjgibson]

Proposed changes

We do something particularly silly, and run a CloudStrike Falcon LogScale instance (nee Humio) behind a Kubernetes Ingress on one of our clusters that we'd like to emit logs to via Fluentd from various other Kubernetes clusters. In order to traverse that Ingress, we need to present a custom Authorization header alongside our requests. This PR adds support for a new configuration parameter, custom_headers, that is used to populate the override_headers Hash used in requests made by this plugin.

For context here, prior to the Elasticsearch 8.0.0 release Humio recommended using Fluentd's elasticsearch output plugin for use in relaying logs for ingest, but now recommends use of this splunk_hec output plugin instead. Said elasticsearch output plugin supported a virtually identical custom_headers configuration block, which we used for this purpose.

It's been ages since I wrote much Ruby anything, but I've attempted to add relevant tests in-line with the existing tests I see in this repository, and make unit-test runs without issue for me - but please let me know if there's something else I should test, or some better way to go around adding this functionality. (I noticed the exposed proxy_from_env hook, but wasn't sure if that could be leveraged for this use-case.)

Types of changes

What types of changes does your code introduce?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING doc
• I have read the CLA
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)
• Any dependent changes have been merged and published in downstream modules

[hvaghani221]

Hey @cjgibson, thank you for your contribution.
I'll try to review your PR next week.

[cjgibson]

Hey @cjgibson, thank you for your contribution.

I'll try to review your PR next week.

No rush - feel free to wait until after the coming holiday season!

[cjgibson]

Hello, hello, checking in again now that the holiday season has wrapped - any chance I could get a review @harshit-splunk?

[cjgibson]

New week, new bump - any ETA here @harshit-splunk ?

[cjgibson]

Thank you @harshit-splunk! Sorry for all the pings 😁

[hvaghani221]

@cjgibson sorry for the delay. We are planning to deprecate this project soon in favour of https://github.com/signalfx/splunk-otel-collector-chart.

[cjgibson]

@cjgibson sorry for the delay. We are planning to deprecate this project soon in favour of https://github.com/signalfx/splunk-otel-collector-chart.

Hahahah - oh no! I'll warn the Humio folks. Does that mean Splunk is dropping support for Fluentd entirely?

[hvaghani221]

Does that mean Splunk is dropping support for Fluentd entirely?

Most probably yes. I'll have to confirm it with the PM.