Skip to content
This repository has been archived by the owner on Nov 24, 2023. It is now read-only.

election: don't forwarding when startLeaderComponent #1118

Merged
merged 14 commits into from
Oct 13, 2020
Merged

election: don't forwarding when startLeaderComponent #1118

merged 14 commits into from
Oct 13, 2020

Conversation

lance6716
Copy link
Collaborator

What problem does this PR solve?

close #1088

What is changed and how it works?

lock more code

Check List

Tests

  • Pass original test

Code changes

Side effects

  • Possible performance regression

Related changes

  • Need to cherry-pick to the release branch

@lance6716 lance6716 added priority/normal Minor change, requires approval from ≥1 primary reviewer status/PTAL This PR is ready for review. Add this label back after committing new changes type/bug-fix Bug fix status/WIP This PR is still work in progress labels Sep 29, 2020
@lance6716 lance6716 added status/WIP This PR is still work in progress and removed status/WIP This PR is still work in progress labels Sep 29, 2020
@lance6716

This comment has been minimized.

Sign in to view
@lance6716
Copy link
Collaborator Author

lance6716 commented Sep 30, 2020 •
edited
Loading

log in #1088

[2020/09/23 15:25:59.801 +08:00] [INFO] [election.go:55] ["current member become the leader"] ["current member"=master2]
...
[2020/09/23 15:26:00.263 +08:00] [INFO] [server.go:2188] [payload="leader:true "] [request=ListMember]
[2020/09/23 15:26:00.264 +08:00] [INFO] [server.go:2203] ["will forward after a short interval"] [from=master2] [to=master3] [request=ListMember]
...
[2020/09/23 15:26:02.518 +08:00] [INFO] [server.go:2188] [payload="leader:true "] [request=ListMember]
[2020/09/23 15:26:02.518 +08:00] [INFO] [server.go:2203] ["will forward after a short interval"] [from=master2] [to=master3] [request=ListMember]
...
[2020/09/23 15:26:04.780 +08:00] [INFO] [server.go:2188] [payload="leader:true "] [request=ListMember]
[2020/09/23 15:26:04.780 +08:00] [INFO] [server.go:2203] ["will forward after a short interval"] [from=master2] [to=master3] [request=ListMember]
...
[2020/09/23 15:26:07.092 +08:00] [INFO] [optimist.go:87] ["the shard DDL optimist has started"] [component="shard DDL optimist"]
[2020/09/23 15:26:07.244 +08:00] [INFO] [server.go:2188] [payload="leader:true "] [request=ListMember]

this PR now will return to dm-ctl an error that failed to forward after waiting 1s. I'm afraid that's too short because in above log master used 7s to finish startLeaderComponent. should we improve waiting time?

and in fact, before this PR, request will forward back and forth and finally success with longer response time. after this PR it will fail quickly. Is this expected?

@lance6716 lance6716 changed the title election: lock more code to reduce forwarding election: don't forwarding when startLeaderComponent Sep 30, 2020
@lance6716 lance6716 removed the status/WIP This PR is still work in progress label Sep 30, 2020
csuzhangxc
csuzhangxc reviewed Sep 30, 2020
View reviewed changes
@csuzhangxc
Copy link
Member

this PR now will return to dm-ctl an error that failed to forward after waiting 1s. I'm afraid that's too short because in above log master used 7s to finish startLeaderComponent. should we improve waiting time?

I think no need to waiting more time, just give a clear error is enough.

and in fact, before this PR, request will forward back and forth and finally success with longer response time. after this PR it will fail quickly. Is this expected?

I think this is expected.

csuzhangxc
csuzhangxc reviewed Oct 9, 2020
View reviewed changes
Copy link
Member

@csuzhangxc csuzhangxc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

csuzhangxc
csuzhangxc reviewed Oct 12, 2020
View reviewed changes
Copy link
Member

@csuzhangxc csuzhangxc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@csuzhangxc csuzhangxc added status/LGT1 One reviewer already commented LGTM and removed status/PTAL This PR is ready for review. Add this label back after committing new changes labels Oct 12, 2020
@csuzhangxc csuzhangxc added this to the v2.0.0 RC.3 milestone Oct 12, 2020
@lance6716
Copy link
Collaborator Author

/run-all-tests

@lance6716 lance6716 requested a review from GMHDBJD October 12, 2020 07:39
@lance6716
Copy link
Collaborator Author

PTAL if have spare time @WangXiangUSTC

WangXiangUSTC
WangXiangUSTC approved these changes Oct 13, 2020
View reviewed changes
Copy link
Contributor

@WangXiangUSTC WangXiangUSTC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lance6716 lance6716 merged commit bbdb8d6 into pingcap:master Oct 13, 2020
@lance6716 lance6716 added the needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 label Oct 13, 2020
@lance6716 lance6716 deleted the fix-1088 branch October 13, 2020 11:04
ti-srebot pushed a commit to ti-srebot/dm that referenced this pull request Oct 13, 2020
@lance6716 @ti-srebot
cherry pick pingcap#1118 to release-2.0
b3a5887 
Signed-off-by: ti-srebot <[email protected]>
@ti-srebot ti-srebot mentioned this pull request Oct 13, 2020
Merged
@ti-srebot
Copy link

cherry pick to release-2.0 in PR #1157

@ti-srebot ti-srebot added already-cherry-pick-2.0 The related PR is already cherry-picked to release-2.0. Add this label once the PR is cherry-picked and removed needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 labels Oct 13, 2020
csuzhangxc pushed a commit that referenced this pull request Oct 14, 2020
@ti-srebot @lance6716
cherry pick #1118 to release-2.0 (#1157)
666a89b 
Signed-off-by: ti-srebot <[email protected]>

Co-authored-by: lance6716 <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@csuzhangxc csuzhangxc csuzhangxc left review comments

@WangXiangUSTC WangXiangUSTC WangXiangUSTC approved these changes

@GMHDBJD GMHDBJD Awaiting requested review from GMHDBJD

Assignees
No one assigned
Labels
already-cherry-pick-2.0 The related PR is already cherry-picked to release-2.0. Add this label once the PR is cherry-picked priority/normal Minor change, requires approval from ≥1 primary reviewer status/LGT1 One reviewer already commented LGTM type/bug-fix Bug fix
Projects
None yet
Milestone
v2.0.0
Development

Successfully merging this pull request may close these issues.

do not forward requests once a DM-master itself become the leader
5 participants
@lance6716 @csuzhangxc @ti-srebot @WangXiangUSTC @GMHDBJD