-
Notifications
You must be signed in to change notification settings - Fork 545
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
validation: allow more labels for _info metrics by default #10028
Conversation
The info metrics are generally low cardinality even though there are many informational labels. This is to support conversion from OTLP attributes easier. Signed-off-by: György Krajcsovits <[email protected]>
Signed-off-by: György Krajcsovits <[email protected]>
Signed-off-by: György Krajcsovits <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please see comments.
|
||
This non-critical error occurs when Mimir receives a write request that contains an info series with a number of labels that exceed the configured limit. | ||
An info series is a series where the metric name ends in `_info`. | ||
The limit protects the system’s stability from potential abuse or mistakes. To configure the limit on a per-tenant basis, use the `-validation.max-label-names-per-info-series` option. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To configure per tenant, you need YAML don't you rather than the flag:
The limit protects the system’s stability from potential abuse or mistakes. To configure the limit on a per-tenant basis, use the `-validation.max-label-names-per-info-series` option. | |
The limit protects the system’s stability from potential abuse or mistakes. To configure the limit on a per-tenant basis, use the `validation.max_label_names_per_info_series` option. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ack, I'll update this and where I copy pasted from above as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also it becomes limits.max_label_names_per_info_series
I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually there's 10 places where we use -validation...
flag and zero where we write yaml option. Do you want to change all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I certainly wouldn't change existing code as part of this PR. I just don't think it makes sense to refer to a flag, when you have to use YAML per-tenant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work, @krajorama . I left a few minor comments and suggestions.
### err-mimir-max-label-names-per-info-series | ||
|
||
This non-critical error occurs when Mimir receives a write request that contains an info series with a number of labels that exceed the configured limit. | ||
An info series is a series where the metric name ends in `_info`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is info series
an abbreviation for information series
or is it really just called an info series
? If it's an abbreviation, we should spell it out except when using a code-formatted label.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not an abbreviation, these are called info series/metrics everywhere
|
||
This non-critical error occurs when Mimir receives a write request that contains an info series with a number of labels that exceed the configured limit. | ||
An info series is a series where the metric name ends in `_info`. | ||
The limit protects the system’s stability from potential abuse or mistakes. To configure the limit on a per-tenant basis, use the `-validation.max-label-names-per-info-series` option. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: I think misuse
has a more positive connotation than abuse
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The doc uses abuse
21 times, and misuse
zero times. Let's change in a different PR if we want to.
Co-authored-by: Arve Knudsen <[email protected]> Co-authored-by: Taylor C <[email protected]>
Signed-off-by: György Krajcsovits <[email protected]>
Signed-off-by: György Krajcsovits <[email protected]>
Could you rebase on main? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you're referencing the wrong flag in the error.
Signed-off-by: György Krajcsovits <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks
What this PR does
The info metrics are generally low cardinality even though there are many informational labels. This is to support conversion from OTLP attributes easier.
Which issue(s) this PR fixes or relates to
Related to prometheus/prometheus#15448
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]
.about-versioning.md
updated with experimental features.