This repository has been archived by the owner on Aug 25, 2021. It is now read-only.
Set check_update_interval for healthchecks #674
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The check_update_interval setting controls how often Consul clients
update Consul servers when the output of their checks have changed but
the status hasn't. This
defaults to 5m because often users would create checks that would run
every minute or so and whose output
contained a timestamp that would change on every iteration of the check.
This would cause a lot of writes to the Consul servers even though
the status of the check didn't change.
In our use-case, we only update the check when the pod's readiness
status changes, so we won't be spamming updates. We run into an issue
though because we always set the check to failing when the pod is first
created and then, because of an issue with Consul where you can't set
the check output on first creation, we issue an update to the check with
the same status (critical) but with output "Pod is pending". This output
will take 5 minutes to show up in the UI without this setting. That
might be okay if the pod immediately becomes healthy but if it doesn't
then it's likely going to be confusing for users why the check is
failing but it doesn't have any output.
Additionally, if the pod's readiness probes fail immediately, then we'll
update the output to be something like "Readiness probes failing" but
again, the UI won't be updated because the status is still critical and
so again the UI would show the check failing with no output.
(also rename the test file to match the file it's testing as per our convention).