Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: ensure exported Prometheus metrics are unique #2173

Merged
merged 7 commits into from
Jul 6, 2023

Conversation

cgrinds
Copy link
Collaborator

@cgrinds cgrinds commented Jun 30, 2023

No description provided.

@Hardikl Hardikl linked an issue Jul 5, 2023 that may be closed by this pull request
// Skip metrics that are belongs to aggr_efficiency template as Rest collector don't have separate template
skip := []string{"logical_used_wo_snapshots", "logical_used_wo_snapshots_flexclones", "physical_used_wo_snapshots", "physical_used_wo_snapshots_flexclones", "total_logical_used", "total_physical_used"}
for _, s := range skip {
if strings.Contains(dupMetric, s) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the metric name be an exact match, or should it support all metrics starting with a specific prefix? While supporting metrics with a prefix can be useful, an exact match is preferable in this case to avoid any potential skipping caused by a "contains" check.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Start with exact and relax if needed. That means this check can change to a map instead of searching a slice.

These are the metrics in question

aggr_logical_used_wo_snapshots
aggr_logical_used_wo_snapshots_flexclones
aggr_physical_used_wo_snapshots
aggr_physical_used_wo_snapshots_flexclones
aggr_total_logical_used
aggr_total_physical_used

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the check to above where key as aggr_total_logical_used{aggr=\"astra_aggr2\",cluster=\"A250-41-42-43\",datacenter=\"Rest\",node=\"A250-42\"} and now HasPrefix would be used instead contains.

sort.Strings(duplicateMetrics)
for _, dupMetric := range duplicateMetrics {
if shouldSkipMetric(dupMetric) {
log.Info().Str("metric", dupMetric).Msg("Skip")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we need to log the skipped metric. If we do, might be better to be explicit with something like "Ignore duplicate"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed as such, changing to trace.

Copy link
Collaborator Author

@cgrinds cgrinds left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Nice job finding this issue @rahulguptajss and @Hardikl for fixing

@rahulguptajss rahulguptajss merged commit a3d37f8 into main Jul 6, 2023
@rahulguptajss rahulguptajss deleted the cbg-ci-unique-metrics branch July 6, 2023 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure Harvest exported Prometheus metrics are unique
3 participants