Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: Priority for load-based rebalancing #34747

Closed
bdarnell opened this issue Feb 8, 2019 · 1 comment
Closed

storage: Priority for load-based rebalancing #34747

bdarnell opened this issue Feb 8, 2019 · 1 comment
Labels
A-kv-distribution Relating to rebalancing and leasing. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@bdarnell
Copy link
Contributor

bdarnell commented Feb 8, 2019

Replica snapshots currently use two priority buckets: recovery (for ranges below their target for live nodes) and rebalance (for all other moves). These are rate-limited to 8MB/s and 2MB/s respectively. Load-based rebalancing should be prioritized higher than count-based rebalancing, and probably use the higher "recovery" rate limit. (within the bucket we can further prioritize based on the degree of imbalance)

@knz knz added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv-distribution Relating to rebalancing and leasing. labels Feb 11, 2019
@bdarnell
Copy link
Contributor Author

The distinction between these two buckets has mostly been eliminated: they still use two separate cluster settings, but the defaults for both are the same, and in general we recommend that when users change one they change both. (and the introduction of learner replicas means that the "rebalancing" bucket is now used much more often than the "recovery" bucket). This kind of coarse-grained prioritization no longer seems desirable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv-distribution Relating to rebalancing and leasing. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Development

No branches or pull requests

2 participants