Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schedulers: decrease target filter calculate #4080

Closed
wants to merge 1 commit into from

Conversation

bufferflies
Copy link
Contributor

What problem does this PR solve?

related #3744,remove some repeat target filter

What is changed and how it works?

scheduler move some target filter such as StoreStateFilter, RegionScoreFilter, SpecialFilter at scheduler start

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)

origin
image

this PR:
image

import double performance

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch

Release note

decrease target filter calculate to improve balance region scheduler  performance

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Sep 6, 2021
@ti-chi-bot ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Sep 15, 2021
Signed-off-by: bufferflies <[email protected]>
@codecov
Copy link

codecov bot commented Oct 11, 2021

Codecov Report

Base: 74.70% // Head: 74.85% // Increases project coverage by +0.15% 🎉

Coverage data is based on head (da90262) compared to base (5e0c212).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4080      +/-   ##
==========================================
+ Coverage   74.70%   74.85%   +0.15%     
==========================================
  Files         260      260              
  Lines       26625    26632       +7     
==========================================
+ Hits        19889    19936      +47     
+ Misses       4950     4923      -27     
+ Partials     1786     1773      -13     
Flag Coverage Δ
unittests 74.85% <100.00%> (+0.15%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/schedule/filter/filters.go 88.16% <100.00%> (-2.53%) ⬇️
server/schedulers/balance_leader.go 91.55% <100.00%> (ø)
server/schedulers/balance_region.go 89.34% <100.00%> (+0.17%) ⬆️
server/schedulers/utils.go 95.23% <100.00%> (+0.01%) ⬆️
pkg/etcdutil/etcdutil.go 84.70% <0.00%> (-3.53%) ⬇️
server/core/hot_region_storage.go 74.68% <0.00%> (-1.27%) ⬇️
server/cluster/cluster.go 82.33% <0.00%> (-0.36%) ⬇️
client/client.go 72.50% <0.00%> (-0.14%) ⬇️
server/grpc_service.go 50.45% <0.00%> (+0.61%) ⬆️
... and 9 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@@ -53,14 +53,17 @@ type balancePlan struct {

sourceScore float64
targetScore float64

excludeTarget []*core.StoreInfo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generation and use of excludeTarget has nothing to do with balancePlan, I think it should be in balance_region.go.

@@ -201,7 +205,7 @@ func (s *balanceRegionScheduler) Schedule(cluster opt.Cluster) []*operator.Opera
continue
}

if op := s.transferPeer(plan); op != nil {
if op := s.transferPeer(plan, stores[index:]); op != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use sorted sourceStores and targetStores and indexSource and indexTarget.

@@ -137,22 +136,24 @@ func (s *balanceRegionScheduler) IsScheduleAllowed(cluster opt.Cluster) bool {
}

func (s *balanceRegionScheduler) Schedule(cluster opt.Cluster) []*operator.Operator {
storeState := &filter.StoreStateFilter{ActionScope: s.GetName(), MoveRegion: true}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be a constant.

filter.NewPlacementSafeguard(s.GetName(), plan.cluster, plan.region, plan.source),
filter.NewRegionScoreFilter(s.GetName(), plan.source, plan.cluster.GetOpts()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is NewRegionScoreFilter now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The score should sort once in source selection, and the targets's score must less than the source by index.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove NewRegionScoreFilter then.

@ti-chi-bot
Copy link
Member

@bufferflies: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 15, 2022
@bufferflies
Copy link
Contributor Author

Outdated, ref #5544

@bufferflies bufferflies deleted the feature/filterOnce branch September 22, 2022 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note Denotes a PR that will be considered when it comes time to generate release notes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants