-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dynamic Shovel deletion: use a more effective way (backport #11321) #11324
Conversation
Cherry-pick of cae964d has failed:
To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally |
bb83dc6
to
c5c4d1d
Compare
Dynamic Shovels keep track of their status in a node-local ETS table which is updated as Shovels go through the lifecycle: start, init (connect, declare the topology), stop. This makes failing Shovels a bit special: their status records will not be long lived, which means it will be considered not to exist by certain code paths. In addition, for such Shovels we do not know what node they are hosted on. But that's fine: we just need to clear their runtime parameter and a periodic Shovel cleanup will remove all children for which no schema database entry (a runtime parameter one) does not exist. rabbitmq_shovel_management's key integration suite has been reworked and expanded to include a case where the Shovel has no chance of successfully connecting. This also deletes a mock-based test suite which does not serve much of a purpose. (cherry picked from commit cae964d) # Conflicts: # deps/rabbitmq_shovel_management/test/http_SUITE.erl # deps/rabbitmq_shovel_management/test/rabbit_shovel_mgmt_SUITE.erl
c5c4d1d
to
11e2307
Compare
I have tested this PR with a three node cluster where two nodes were RabbitMQ |
1. There is no need to test Shovel deletion in this specific test 2. When a Shovel is starting, it cannot be deleted with several RabbitMQ versions due to [A] A. rabbitmq/rabbitmq-server#11324
Dynamic Shovels keep track of their status in
a node-local ETS table which is updated as
Shovels go through the lifecycle: start,
init (connect, declare the topology), stop.
This makes failing Shovels a bit special:
their status records will not be long lived,
which means it will be considered not to exist
by certain code paths.
In addition, for such Shovels we do not know what
node they are hosted on. But that's fine:
we just need to clear their runtime parameter
and a periodic Shovel cleanup will remove all
children for which no schema database entry
(a runtime parameter one) does not exist.
References #4242.
This is an automatic backport of pull request #11321 done by Mergify.