Use new rabbitmqctl features for monitoring #916
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| if [ "$rc_timeouts" -eq 2 ]; then | ||
| master_score 0 | ||
| return $OCF_ERR_GENERIC | ||
| elif [ $rc -ne 0 ]; then |
There was a problem hiding this comment.
Here should be something like (rc_timeouts == 0 AND rc != 0) because if (rc_timeouts == 1 AND rc == 137), that is a timeout which should be ignored
| stopped/demoted. | ||
| </longdesc> | ||
| <shortdesc lang="en">Use --local option for list_queues</shortdesc> | ||
| <content type="string" default="${OCF_RESKEY_rmq_feature_local_list_queues_default}" /> |
There was a problem hiding this comment.
Here and above you can use type="boolean", like it is done for 'debug' parameter, for example. Though I am not sure if it will affect anything at all.
|
@binarin: overall patch looks good to me, it looks like it should remove load done by our current monitoring. |
binarin
force-pushed
the
rabbitmq-server-new-shiny-ocf-health-check
branch
from
a81272b to
464f54b
Compare
|
What's the conclusion on this? Should we merge it? |
|
+1, but please remove DO NOT MERGE if you think it's done and it works for you |
|
@michaelklishin Please don't merge it yet, I'm still testing. |
|
This currently doesn't merge cleanly. |
This will stop wasting network bandwidth for monitoring. E.g. a 200-node OpenStack installation produces aronud 10k queues and 10k channels. Doing single list_queues/list_channels in cluster in this environment results in 27k TCP packets and around 12 megabytes of network traffic. Given that this calls happen ~10 times a minute with 3 controllers, it results in pretty significant overhead. To enable those features you shoud have rabbitmq containing following patches: - rabbitmq#883 - rabbitmq#911 - rabbitmq#915
binarin
force-pushed
the
rabbitmq-server-new-shiny-ocf-health-check
branch
from
464f54b to
99f2a48
Compare
binarin
changed the title
|
Rebased and tested again. Now I'm happy with this patch. |
|
Thank you! |
openstack-gerrit
pushed a commit
to openstack-archive/fuel-library
that referenced
this pull request
Aug 24, 2016
This will stop wasting network bandwidth for monitoring. E.g. a 200-node OpenStack installation produces aronud 10k queues and 10k channels. Doing single list_queues/list_channels in cluster in this environment results in 27k TCP packets and around 12 megabytes of network traffic. Given that this calls happen ~10 times a minute with 3 controllers, it results in pretty significant overhead. Upstream change: - rabbitmq/rabbitmq-server#916 To enable those features you shoud have rabbitmq containing following patches: - rabbitmq/rabbitmq-server#883 - rabbitmq/rabbitmq-server#911 - rabbitmq/rabbitmq-server#915 Change-Id: Icfde3360b42a841ad3a219b94f65a69b2a18cea7 Closes-Bug: 1614071
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@dmitrymex @bogdando WDYT? I haven't tested it yet, but if you are OK with overall shape of this patch, I'll start polishing and testing it.
To stop wasting network bandwidth during health checks (e.g. list_queues
in 3-node cluster with 10k queues costs on average 12 megabytes of
traffic and 27k TCP packets).
Features are disabled by default to preserve compatibility, but they
SHOULD be enabled when following patches are present in currently used
rabbitmq version:
node_health_checkcompletely node-local #883