Update durable queues outside of Khepri transactions #10742
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`rabbit_db_queue:update_durable/2`'s caller (`rabbit_amqqueue:mark_local_durable_queues_stopped`/1) passes a filter function that performs some operations that aren't allowed within Khepri transactions like looking up and using the current node and executing an RPC. Calling `rabbit_amqqueue:mark_local_durable_queues_stopped/1` on a Rabbit with the `khepri_db` feature flag enabled will result in an error. We can safely update a number of queues by using Khepri's `khepri_adv:get_many/3` advanced API which returns the internal version number of each queue. We can filter and update the queues outside of a transaction function and then perform all updates at once, failing if any queue has changed since the `khepri_adv:get_many/3` query. So we get the main benefits of a transaction but we can still execute any update or filter function.
the-mikedavis
force-pushed
the
md-fix-vhost-recovery-queue-update
branch
from
0ade88e to
8a03b28
Compare
|
Thanks, I can no longer reproduce the problem. |
the-mikedavis
added a commit
that referenced
this pull request
Mar 20, 2024
Update durable queues outside of Khepri transactions (backport #10742)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@mkuratczyk found a bug that happens when the
khepri_dbfeature flag is enabled and a vhost fails to recover. Specifically this block:rabbitmq-server/deps/rabbit/src/rabbit_vhost_process.erl
Lines 49 to 53 in e8e50c8
The
rabbit_amqqueue:mark_local_durable_queues_stopped/1call fails when Khepri is enabled because it tries to create a transaction function which Khepri disallows - it finds queues which are not alive (via an RPC call) and marks them as stopped. That's unsafe to do in a Khepri transaction since it would be executed on each node and the value ofnode()would change, and side effects like RPC calls would be repeated by each Khepri cluster member. So that function currently errors. (For example start a 3.13.0 broker, enablekhepri_dband executerabbit_amqqueue:mark_local_durable_queues_stopped(<<"/">>)- this will error out).We can work around this by using Khepri's advanced API to get the version number in the database of each queue, filter and update each queue and then use a transaction to apply the updates. This way we get transaction-like behavior of atomically updating all queues or none without the restrictions of Khepri transaction functions for FilterFun and UpdateFun.