-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: send/recv: support for additional features in backup streams #14997
base: master
Are you sure you want to change the base?
Conversation
d99ffcc
to
e0df20a
Compare
Someone pointed out "DMU features and SPA features may not always line up, so we'll need to support both kinds". Of course I knew this, and I had gone back and forth a couple of times on how to do it, and then decided I was getting into the weeds for a proof of concept and so went with the dumbest thing I could imagine, which was in fact too dumb. Update pushed, original description edited. |
Since we're out of DMU_BACKUP_FEATURE_* flags, we need another way to signal required features to a reciever. This takes the last remaining feature flag as DMU_BACKUP_FEATURE_EXT_FEATURES, and uses it to signal that additional features are listed in the "features" key in the nvlist in the stream's BEGIN record. The conception here is very simple. If the stream says it needs a feature, then the recieving end must support that that feature in order to recieve it. What "support" means will depend on the feature, but at least it starts with the checks in recv_begin_check_feature_flags_impl().
e0df20a
to
d98948f
Compare
I'm not opposed to this idea in general, though I do think that we could probably just steal some or all of the reserved bits in the remainder of the drr_versioninfo struct. I don't know of any proposed use for them. I also don't know what the reserved Delphix flag is for; I don't believe we have any deployed changes that make use of it, though I would want to check. That might have been for a precursor to redacted send/recv? |
Mm, you might be right. Of course, its just kicking the can down the road, but its a lot more road so maybe that's ok! Seems like they've been marked "reserved" since the beginning (illumos/illumos-gate@9e69d7d) so probably its just fine. I'd like to know the thought behind reserving them in the first place, but I doubt it makes much difference. I'll make another PR to open these flags up and see how that goes (probably 99% doc patch, but I'll check the code too).
I'd love to know (and the Nutanix feature too), but its not critical. |
Oh huh, also:
which explains why bit 29 is the "last flag", but I can't find anything about why 30 and 31 aren't used. I love mysteries :) |
Oh, right:
|
Motivation and Context
I'm currently working on a compression option for a customer. I'm not yet sure if it will pan out, but if it does, its likely to require a stream feature flag to keep send/recv working. However, we're out of
DMU_BACKUP_FEATURE_*
bits, so we need some other mechanism for adding more features. This PR adds such a mechanism.This PR is meant as a starting point for discussion. Please comment!
Description
This takes the last remaining feature flag and names it
DMU_BACKUP_FEATURE_EXT_FEATURES
. That flag is used to signal that there is afeatures
key in the nvlist in the stream's BEGIN record. In there, is an nvlist that lists all the features required to recieve the stream.The conception here is very simple. If the stream says it needs a feature, then the receiving end must support that that feature in order to recieve it. What "support" means will depend on the feature, but at least it starts with the checks in
recv_begin_check_feature_flags_impl()
.The values are ignored. I can imagine them being a good spot for providing parameters for the named feature, but for now simple flags will do.
This PR includes a simple per-dataset feature,
com.despairlabs:fancy_butter
that does nothing except change the enabled/activated state when a corresponding property is enabled on the dataset. Obviously, I don't expect this to be included; I just need something to show how it hooks up (seriously though: fancy butters are awesome).I've set up
recv_begin_check_feature_flags_impl()
to check that the requested features are enabled on the pool, and added something tosetup_featureflags()
to show how they might be automatically added to the stream, at least for ones with matching SPA features. Quite possibly its better done feature-by-feature; I just wanted to show a possibility.If its useful, here's the original features proposal, that includes streams: https://marc.info/?l=zfs-discuss&m=130635021411635. I did this before I read it, and its pretty close.
Just to prove it, here's some test output:
I think that's it? Not much more to it, just a place to stash more strings. Tell me what you think, and if its worth polishing up, or if there should be more in it. Cheers!
How Has This Been Tested?
Only light manual testing. I am not expecting the test suite to pass.
Types of changes
Checklist:
Signed-off-by
.