-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increase splunkObservability and splunkPlatform timeouts to 20s #438
Conversation
…splunkPlatform timeouts to 20s
@@ -146,6 +146,9 @@ | |||
"profilingEnabled": { | |||
"description": "Whether profiling data is enabled for Splunk Observability", | |||
"type": "boolean" | |||
}, | |||
"timeout": { | |||
"type": "string" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mind adding a description?
@@ -108,6 +108,9 @@ splunkObservability: | |||
# If you don't use AlwaysOn Profiling for Splunk APM, you can disable it. | |||
profilingEnabled: false | |||
|
|||
# HTTP timeout for sending data. | |||
timeout: 20s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
20s is pretty high, so maybe we should keep the default of 10 with the new value until/unless we receive reports of it being exceeded (wasn't the linked issue just for exceeded platform timeouts)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with Ryan. 20s for HTTP connection is too high. We shouldn't change it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your input.
I rarely see the same context error in my k8s clusters that upload data to Splunk Observability, it probably only happens when backend or middle issues are present.
I'm okay with not adding a splunkObservability.timeout config, I thought it might be useful for Observability users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's reasonable option to have. We should merge open-telemetry/opentelemetry-collector-contrib#6803 and expose full set of http settings instead. 10s timeout is already very high. I doubt that users really need it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to block this PR for now. But I'm open to accept it if someone can address my concerns
I just realized that all the http settings that will be added in open-telemetry/opentelemetry-collector-contrib#6803 are squashed, so this config option won't break it. Probably we can add timeout as done in this PR if it's really needed, but I'm apposed to bumping the defaults unless we are confident that this is needed |
I'm going to close this PR for now since we should take a different approach to addressing the related issue. |
Related Issue: signalfx/splunk-otel-collector#1454