Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only allow one transient "tick" process per quorum queue #7389

Closed
kjnilsson opened this issue Feb 22, 2023 · 1 comment
Closed

Only allow one transient "tick" process per quorum queue #7389

kjnilsson opened this issue Feb 22, 2023 · 1 comment
Milestone

Comments

@kjnilsson
Copy link
Contributor

Quorum queues execute metrics emission and other periodic tasks by default every 5000ms. This is done in a separate one-off process. Some of the work done queries the queue process itself as well as each node in the quorum queue membership. If this takes more than the tick interval we may end up running multiple of these tick processes as the same time. During times of great load this could potentially cause runaway process use when there are lots of quorum queues in a system.

We should find a way to ensure each queue only ever runs a single of these tick processes at any one time. Possibly by using node local locking, aux state or name registry.

@SimonUnge
Copy link
Member

Would a similar approach as rabbit_stream_coordinator aux approach suffice, by keeping track of the pid and when the pid dies?

SimonUnge added a commit to SimonUnge/rabbitmq-server that referenced this issue Mar 18, 2023
SimonUnge added a commit to SimonUnge/rabbitmq-server that referenced this issue Mar 20, 2023
SimonUnge added a commit to SimonUnge/rabbitmq-server that referenced this issue Mar 20, 2023
SimonUnge added a commit to SimonUnge/rabbitmq-server that referenced this issue Mar 20, 2023
SimonUnge added a commit to SimonUnge/rabbitmq-server that referenced this issue Mar 21, 2023
mergify bot pushed a commit that referenced this issue Mar 30, 2023
michaelklishin pushed a commit that referenced this issue Mar 30, 2023
mergify bot pushed a commit that referenced this issue May 23, 2023
@michaelklishin michaelklishin added this to the 3.12.0 milestone May 23, 2023
michaelklishin added a commit that referenced this issue May 23, 2023
See #7389. One tick process per quorum queue (backport #7668)
@michaelklishin michaelklishin modified the milestones: 3.12.0, 3.11.17 May 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants