-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add recording rules for Alertmanager dashboard, #387
Conversation
af439cf
to
8ef58d4
Compare
c33b542
to
5cb38bc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - but have one question.
@@ -11,11 +11,11 @@ local utils = import 'mixin-utils/utils.libsonnet'; | |||
}) | |||
.addPanel( | |||
$.panel('Total Alerts') + | |||
$.statPanel('sum(cortex_alertmanager_alerts{%s})' % $.jobMatcher('alertmanager'), format='short') | |||
$.statPanel('sum(cluster_job_%s:cortex_alertmanager_alerts:sum{%s})' % [$._config.per_instance_label, $.jobMatcher('alertmanager')], format='short') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the recording rule is a sum, do we need to sum here again?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question, I also wondered about this, but in general I decided to leave all the sum()s as-is because the dashboard allows selecting multiple cluster/namespaces.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
With large numbers of tenants the queries for some panels on thos dashboard can become quite slow as the metrics exposed are per-tenant.
cd4fb65
to
826af4d
Compare
…ations-rules Add recording rules for Alertmanager dashboard,
What this PR does:
With large numbers of tenants the queries for these panels can become quite
slow as the metrics exposed are per-tenant.
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]