Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tidb] PD failover count throws errors even when there is no error #133

Closed
2 of 10 tasks
unw9527 opened this issue Jun 30, 2022 · 4 comments
Closed
2 of 10 tasks

[tidb] PD failover count throws errors even when there is no error #133

unw9527 opened this issue Jun 30, 2022 · 4 comments
Assignees
Labels
bug Something isn't working

Comments

@unw9527
Copy link
Contributor

unw9527 commented Jun 30, 2022

What problem does this PR solve?

When failovering a broken PD member, this line gives us an error even if no PD member fails (i.e. pdDeletedFailureReplicas=0). When we inspect the error log, we find that the error message "PD failover replicas (0) reaches the limit (0), skip failover" is constantly written to the operator log and is very confusing to user. However, there is actually no error if no PD member fails. We believe when pdDeletedFailureReplicas=0, the operator should not throw the error message "PD failover replicas (0) reaches the limit (0), skip failover".

This PR provides a straightforward fix for this problem.

What is changed and how does it work?

We fix it by adding one more condition in the if statement in this line to check whether pdDeletedFailureReplicas is larger than 0, as illustrated below:

if pdDeletedFailureReplicas >= *tc.Spec.PD.MaxFailoverCount && pdDeletedFailureReplicas > 0 {
	klog.Errorf("PD failover replicas (%d) reaches the limit (%d), skip failover", pdDeletedFailureReplicas, *tc.Spec.PD.MaxFailoverCount)
	return nil
}

Code changes

  • Has Go code change
  • Has CI related scripts change

Tests

  • Unit test
  • E2E test
  • Manual test
  • No code

This is a simple fix and we suppose no test above is needed.

Side effects

  • Breaking backward compatibility
  • Other side effects:

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

NONE
@unw9527 unw9527 added the bug Something isn't working label Jun 30, 2022
@unw9527
Copy link
Contributor Author

unw9527 commented Jun 30, 2022

PR created: pingcap/tidb-operator#4608

@tianyin
Copy link
Member

tianyin commented Jul 1, 2022

@unw9527 you are a PR expert now.

@unw9527
Copy link
Contributor Author

unw9527 commented Jul 10, 2022

Confirmed: pingcap/tidb-operator#4608 (review)

@unw9527 unw9527 closed this as completed Jul 10, 2022
@tylergu
Copy link
Member

tylergu commented Aug 24, 2022

Fixed by pingcap/tidb-operator#4608

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants