Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Khepri to 0.17.0 #12753

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Bump Khepri to 0.17.0 #12753

wants to merge 5 commits into from

Conversation

the-mikedavis
Copy link
Member

This is not ready to be merged yet, Khepri 0.17.0 hasn't yet been released. (And we want to make more changes before then anyways.)

This PR handles the breaking changes in Khepri 0.17.0.

@the-mikedavis the-mikedavis self-assigned this Nov 18, 2024
@the-mikedavis the-mikedavis force-pushed the md/khepri-0-17 branch 3 times, most recently from 6d289fe to cc6185d Compare November 25, 2024 16:28
@the-mikedavis the-mikedavis force-pushed the md/khepri-0-17 branch 6 times, most recently from 5cb0a79 to 42f571a Compare December 6, 2024 20:49
@dumbbell
Copy link
Member

Thank you for updating the branch!

I’m working on updating Ra in Khepri but there are small changes of behaviors that cause test flakes. I fixed one of them (Ra can reture {error, normal} after a Ra server exited), but not the second one yet (that new error causes a retry loop to retry until timeout).

@dumbbell
Copy link
Member

I fixed the bug in khepri_cluster.erl and merged the bump of Ra to 2.16.0.

Do you think we can update Khepri in RabbitMQ now? Perhaps we should focus on that and release Khepri 0.17.0 this week.

the-mikedavis and others added 5 commits January 29, 2025 10:42
`locally_known_members/1` and `locally_known_node/1` were replaced with
`members/2` and `nodes/2` with `favor` set to `low_latency` - this
matches the interface for queries in Khepri.
All callers of `khepri_adv` and `khepri_tx_adv` need updates to handle
the now consistent return type of `khepri:node_props_map()` in Khepri
0.17.

We don't need any compatibility code to handle "either the old return
type or the new return type" because the translation is done entirely
in the "client side" code in Khepri - meaning that the return value from
the Ra server is the same but it is translated differently by the
functions in `khepri_adv` and `khepri_tx_adv`.
If the tree node to delete does not exist, the node props map is empty.
Comment on lines +1120 to +1121
VersionedPath = khepri_path:combine_with_conditions(
Path, [#if_payload_version{version = Vsn}]),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s a good idea to use the payload version here, given the read is outside of the transaction!

But now, I wonder if we need the transaction at all :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the transaction would only be necessary for the sake of performance - it's faster to send the one transaction command than many small delete commands (since Ra handles them in serial). We might be able to make it work with async commands though, or a batch API.

The hope with some of the changes in 0.17 is to remove the transaction for regular queue deletion too (rather than just the transient ones here) since that will save resources + be faster

Comment on lines +714 to +716
case {Path, Props} of
{?RABBITMQ_KHEPRI_EXCHANGE_PATH(VHostName, _),
#{data := X}} ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly a note to myself: I wonder if ?RABBITMQ_KHEPRI_EXCHANGE_PATH() can be use in the function clause directly instead of having a case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hopeful we could do that but we would need https://github.com/erlang/eep/blob/master/eeps/eep-0055.md to be implemented - we would want to pin bindings like ^VHostName. Without EEP55 I believe we can't do that, it would be a compile error to re-use the binding if I remember correctly

@acogoluegnes
Copy link
Contributor

The upgrade to RA 2.16 in the broker seems to cause a flake (#13172) when the stream plugin uses a consistent read to check if some nodes are in maintenance node. Feel free to ping me when you think this PR is ready, I can run the stream plugin test suite against it to make sure the flake is gone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

3 participants