Skip to content
This repository has been archived by the owner on Nov 17, 2020. It is now read-only.

Add worker_pool:dispatch_sync function #368

Merged
merged 3 commits into from
Mar 24, 2020

Conversation

HoloRin
Copy link
Contributor

@HoloRin HoloRin commented Mar 20, 2020

dispatch_sync sits in between the behavior of submit and submit_async, blocking the caller until a worker begins the task (submit blocks until the task is scheduled and complete, while submit_async does not block at all).

This is useful when you want to throttle submissions to the pool from a single process, such that all workers are busy, but there exists no backlog of work for the pool.

HoloRin added a commit to rabbitmq/rabbitmq-server that referenced this pull request Mar 20, 2020
Make use of the new dispatch_sync function in
rabbitmq/rabbitmq-common#368 to block only when all
workers are busy
dispatch_sync sits inbetween the behavior of submit and submit_async,
blocking the caller until a worker begins the task, as opposed
to not blocking at all, or blocking util the task has finished.
This is useful when you want to throttle submissions to the pool
from a single process, such that all workers are busy, but there
exists no backlog of work for the pool.
@HoloRin HoloRin force-pushed the worker_pool_dispatch_sync branch from a9e6849 to 150f8b5 Compare March 20, 2020 14:22
Copy link
Member

@michaelklishin michaelklishin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already a worker pool suite in the server repo. These tests should be moved there or that suite should be moved to this repo but we should not keep both.

@HoloRin
Copy link
Contributor Author

HoloRin commented Mar 23, 2020

There is already a worker pool suite in the server repo. These tests should be moved there or that suite should be moved to this repo but we should not keep both.

I'll move that suite here, since I think next to it's implementation is a more obvious location (I looked for the suite before creating this one, but didn't find it).

HoloRin added 2 commits March 23, 2020 15:26
according to Michael's suggestion
to consolidate worker pool tests in a single location
@michaelklishin michaelklishin merged commit a1e1e0b into master Mar 24, 2020
@michaelklishin michaelklishin deleted the worker_pool_dispatch_sync branch March 24, 2020 21:55
michaelklishin added a commit that referenced this pull request Mar 24, 2020
@michaelklishin
Copy link
Member

The new tests depend on rabbit_ct_helpers which was not listed as a test dependency in this repo, hence the CI failures. It's been added in master.

@HoloRin
Copy link
Contributor Author

HoloRin commented Mar 25, 2020

Thanks @michaelklishin

@gerhard gerhard restored the worker_pool_dispatch_sync branch March 25, 2020 21:39
@gerhard
Copy link
Contributor

gerhard commented Mar 25, 2020

@michaelklishin I need the worker_pool_dispatch_sync branch & startup_memory_fix from rabbitmq/rabbitmq-server#2279 for rabbitmq/tgir#8. Let's keep them around for the duration of TGIR Season 1 which runs through to 25th December 2020. Thanks!

@dumbbell dumbbell added this to the 3.8.4 milestone May 13, 2020
dumbbell pushed a commit to rabbitmq/rabbitmq-server that referenced this pull request May 13, 2020
Make use of the new dispatch_sync function in
rabbitmq/rabbitmq-common#368 to block only when all
workers are busy

(cherry picked from commit 01270fd)
@dumbbell
Copy link
Member

Backported to 3.8.x as part of #387.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants