release-21.1: kv: deflake multi-node kvnemesis #63403
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These were added in b15e3dd, but that was incorrect. It is valid for a merge that races with another merge or split to hit these case, as the Subsume request is non-transactional. Also, allow `RHS range bounds do not match` errors in kvnemesis. This is a valid error for the reasons mentioned above.
Without it, we see the following errors in kvnemesis:
```
Wraps: (2) error applying x.AdminChangeReplicas(ctx, /Table/50/"244956da", [{ADD_VOTER n2,s2}]) // [n1,s1,r36/1:/Table/{40-50/"ba86b…}]: failed to generate LEARNER_INITIAL snapshot: couldn't find range descriptor
```
Fixes cockroachdb#61322. This deflakes TestKVNemesisMultiNode. The test had recently gotten more flaky, which I bisected back to 3fe1992. It appears that the changes in that commit made it more likely to hit this `replica not found in RangeDescriptor`, which is returned after a TransferLease request has acquired latches, in addition to the existing `unable to find store \d+ in range` error, which is returned before a TransferLease request has acquired latches.
|
This change is |
aayushshah15
approved these changes
Apr 9, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 3/3 commits from #62580.
/cc @cockroachdb/release
Fixes #61322.
Prep to address #59062.
kv: don't consider Subsume error as assertion failures
These were added in b15e3dd, but that was incorrect. It is valid for a merge that races with another merge or split to hit these case, as the Subsume request is non-transactional.
Also, allow
RHS range bounds do not matcherrors in kvnemesis. This is a valid error for the reasons mentioned above.kv: mark error from Replica.GetSnapshot with errMarkSnapshotError
Without it, we see the following errors in kvnemesis:
kv: allow two more errors with TransferLeaseOperation
This deflakes TestKVNemesisMultiNode. The test had recently gotten flakier, which I bisected back to 3fe1992. It appears that the changes in that commit made it more likely to hit this
replica not found in RangeDescriptor, which is returned after a TransferLease request has acquired latches, in addition to the existingunable to find store \d+ in rangeerror, which is returned before a TransferLease request has acquired latches.