release-20.2: storage: report all encountered intents in sst export error #64931
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
aliher1911
force-pushed
the
backport20.2-64131
branch
6 times, most recently
from
d918f4e to
d0fadfe
Compare
aliher1911
requested review from
a team and
adityamaru
and removed request for
a team and
adityamaru
aliher1911
force-pushed
the
backport20.2-64131
branch
from
d0fadfe to
18e6479
Compare
Previously, pebbleExportToSst was stopping upon encountering first intent. This was causing backups to be very slow if lots of intents build up. To be able to proceed with export, intent needs to be resolved and export retried. The result of this behaviour is that export would run as many times as there were intents in the table before succeeding. To address this, all intents from the range are collected and reported in WriteIntentError. They could be resolved efficiently as batch similar to how GC operates. Release note (bug fix): Backup no longer resolves intents one by one. This eliminates running a high pri query to cleanup intents to unblock backup in case of intent buildup.
aliher1911
force-pushed
the
backport20.2-64131
branch
from
18e6479 to
7a5a92f
Compare
|
This fix only applies to pebble, but not to rocksdb. |
nvanbenschoten
approved these changes
May 12, 2021
There was a problem hiding this comment.
Reviewed 7 of 7 files at r1.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @aliher1911)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #64131.
/cc @cockroachdb/release
Previously, pebbleExportToSst was stopping upon encountering first
intent.
This was causing backups to be very slow if lots of intents build up.
To be able to proceed with export, intent needs to be resolved and
export retried. The result of this behaviour is that export would run
as many times as there were intents in the table before succeeding.
To address this, all intents from the range are collected and reported
in WriteIntentError. They could be resolved efficiently as batch
similar to how GC operates.
Fixes #59704
Release note (bug fix): Backup no longer resolves intents one by one.
This eliminates running a high pri query to cleanup intents to unblock
backup in case of intent buildup.