-
Notifications
You must be signed in to change notification settings - Fork 183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(metrics): don't forward duplicate metrics from annotated Pods #3171
Conversation
b31abb6
to
d957829
Compare
d957829
to
bbf017a
Compare
@@ -38,7 +38,7 @@ processors: | |||
metrics: | |||
metric: | |||
# we let the metrics from annotations ("kubernetes-pods") through as they are |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# we let the metrics from annotations ("kubernetes-pods") through as they are | |
# we let the metrics from annotations ("pod-annotations") through as they are |
internal.ReceiverMockNamespace, | ||
internal.ReceiverMockServiceName, | ||
internal.ReceiverMockServicePort, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are not going to check receiver-mock metrics anymore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't change any of the test checks, I just moved this logic to a separate function which gets these values from module constants.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do want to get rid of checking for receiver-mock metrics, because they're annoying to work with - receiver-mock only emits some of them based on ingested data, so they're different for each test - but I didn't do that in this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Going to merge it in order to put into the release
Don't forward App metrics if they originate from a Pod scraped due to annotations. We tell customers to do this for these metrics (https://help.sumologic.com/docs/integrations/web-servers/nginx/#configure-metrics-collection), but we have separate remote writes for these cases, and so they are forwarded twice, and with different labels.
I ended up making a bunch of additional changes:
Checklist