Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scheduler: make history-sample-interval and history-sample-duration configurable #7878

Merged
merged 6 commits into from
Mar 7, 2024
Merged

scheduler: make history-sample-interval and history-sample-duration configurable #7878

merged 6 commits into from
Mar 7, 2024

Conversation

lhy1024
Copy link
Contributor

@lhy1024 lhy1024 commented Mar 5, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: Close #7877

What is changed and how does it work?

Check List

Tests

  • Unit test

  • Integration test

  • Manual test (add detailed scripts or steps below)

  • Has the configuration change

Release note

None.

@lhy1024 lhy1024 requested a review from HuSharp March 5, 2024 09:10
@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 5, 2024 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • HuSharp
  • rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 5, 2024
@lhy1024 lhy1024 changed the title scheduler: make history-sample-interval and history-sample-duration c… scheduler: make history-sample-interval and history-sample-duration configurable Mar 5, 2024
@codecov Codecov
Copy link

codecov bot commented Mar 5, 2024 •
edited
Loading

Codecov Report

Merging #7878 (5027f78) into master (57cd603) will increase coverage by 0.04%.
The diff coverage is 89.36%.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #7878      +/-   ##
==========================================
+ Coverage   73.39%   73.44%   +0.04%     
==========================================
  Files         434      434              
  Lines       48035    48067      +32     
==========================================
+ Hits        35257    35302      +45     
+ Misses       9725     9716       -9     
+ Partials     3053     3049       -4
Flag Coverage Δ
unittests 73.44% <89.36%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

rleungx
rleungx reviewed Mar 5, 2024
View reviewed changes
pkg/schedule/schedulers/hot_region_config.go
@@ -147,6 +153,9 @@ type hotRegionSchedulerConfig struct {
ForbidRWType string `json:"forbid-rw-type,omitempty"`
// SplitThresholds is the threshold to split hot region if the first priority flow of on hot region exceeds it.
SplitThresholds float64 `json:"split-thresholds"`

HistorySampleDuration typeutil.Duration `json:"history-sample-duration"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have too many hot scheduler configs. Can we make some of them unexposed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course. I will hidden some config in another pr.

lhy1024 added 2 commits March 6, 2024 14:18
@lhy1024
add more test
19e509b 
Signed-off-by: lhy1024 <[email protected]>
@lhy1024
allow windows is zero
200d65e 
Signed-off-by: lhy1024 <[email protected]>
HuSharp
HuSharp reviewed Mar 6, 2024
View reviewed changes
pkg/statistics/store_load.go
if s.sampleDuration == sampleDuration && s.sampleInterval == sampleInterval {
return s
}
return NewStoreHistoryLoads(s.dim, sampleDuration, sampleInterval)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to save previous loads?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not necessary and it will make code complex

@HuSharp
Copy link
Member

HuSharp commented Mar 7, 2024 •
edited
Loading

tested
image

HuSharp
HuSharp approved these changes Mar 7, 2024
View reviewed changes
Copy link
Member

@HuSharp HuSharp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ci failed :)

@ti-chi-bot ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 7, 2024
@ti-chi-bot ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 7, 2024
@lhy1024 lhy1024 added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Mar 7, 2024
@lhy1024
Copy link
Contributor Author

lhy1024 commented Mar 7, 2024

/merge

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 7, 2024

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 7, 2024

This pull request has been accepted and is ready to merge.

Commit hash: 5027f78

@ti-chi-bot ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 7, 2024
@ti-chi-bot ti-chi-bot bot merged commit bbd3bdb into tikv:master Mar 7, 2024
25 of 26 checks passed
ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Mar 7, 2024
@lhy1024 @ti-chi-bot
This is an automated cherry-pick of tikv#7878
e6c2318 
close tikv#7877

Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #7893.

@lhy1024 lhy1024 deleted the history-config branch March 7, 2024 06:34
@ti-chi-bot ti-chi-bot mentioned this pull request Mar 7, 2024
Closed
@ti-chi-bot ti-chi-bot removed the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HuSharp HuSharp HuSharp approved these changes

@rleungx rleungx rleungx approved these changes

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

dynamic change history load windows length
4 participants
@lhy1024 @HuSharp @ti-chi-bot @rleungx