Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver,roachpb,storage: add IsSpanEmptyRequest to check for any data #85798

Merged
merged 1 commit into from
Aug 9, 2022

Conversation

ajwerner
Copy link
Contributor

@ajwerner ajwerner commented Aug 9, 2022

This commit adds a new request, IsSpanEmptyRequest, which checks to see if
there is any data in a key span of any kind. It ignore the GC threshold, and
operates across all versions.

Fixes #85726

Release note: None

@ajwerner ajwerner requested review from a team as code owners August 9, 2022 04:37
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@ajwerner ajwerner marked this pull request as draft August 9, 2022 04:37
Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, thanks!

Reviewed 12 of 12 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner)


pkg/kv/kvserver/batcheval/cmd_is_span_empty_test.go line 56 at r1 (raw file):

	checkIsEmpty := func(t *testing.T, exp bool, from, to roachpb.Key) {
		var ba kv.Batch
		ba.Header.MaxSpanRequestKeys = 1

This is the only reason the DistSender isn't parallelizing the request. So if we don't set this, we'll get back the number of non-empty ranges, scanned in parallel. I suppose that might make sense, in a way, and we could extend it to also count the number of keys (I think David mentioned wanting this for some purpose). We should mention this in the request comment.


pkg/roachpb/api.proto line 203 at r1 (raw file):

}

// IsSpanEmptyResponse is the response to an IsSpanEmptyRequest.

Let's mention that NumKeys is what callers want.


pkg/storage/mvcc.go line 5285 at r1 (raw file):

	// If we're not exporting all revisions then we can mask point keys below any
	// MVCC range tombstones, since we don't care about them.
	var rangeKeyMasking hlc.Timestamp

We don't really need this.


pkg/storage/mvcc.go line 5294 at r1 (raw file):

		EndTime:              opts.EndTS,
		RangeKeyMaskingBelow: rangeKeyMasking,
		IntentPolicy:         MVCCIncrementalIterIntentPolicyEmit,

Let's surface inline values too, rather than error on them.


pkg/storage/testdata/mvcc_histories/clear_range line 41 at r1 (raw file):


run ok
is_span_empty k=a end=+a

We could have some more test cases here (e.g. range keys, time spans etc), but no need to spend time on that now. The implementation is trivial, and MVCCIncrementalIterator itself has plenty of coverage.

@ajwerner ajwerner force-pushed the ajwerner/is-span-empty-request branch from 64f578d to c05cca8 Compare August 9, 2022 13:21
Copy link
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner and @erikgrinaker)


pkg/kv/kvserver/batcheval/cmd_is_span_empty_test.go line 56 at r1 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

This is the only reason the DistSender isn't parallelizing the request. So if we don't set this, we'll get back the number of non-empty ranges, scanned in parallel. I suppose that might make sense, in a way, and we could extend it to also count the number of keys (I think David mentioned wanting this for some purpose). We should mention this in the request comment.

Yeah, correct. One thing I did is changed the combine method to set the NumKeys back to 1.


pkg/roachpb/api.proto line 203 at r1 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

Let's mention that NumKeys is what callers want.

Done.


pkg/storage/mvcc.go line 5281 at r1 (raw file):

	var span *tracing.Span
	ctx, span = tracing.ChildSpan(ctx, "MVCCIsSpanEmpty")
	defer span.Finish()

I removed this tracing span too; don't think we need it

Code quote:

	ctx, span = tracing.ChildSpan(ctx, "MVCCIsSpanEmpty")
	defer span.Finish()

pkg/storage/mvcc.go line 5285 at r1 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

We don't really need this.

copy-pasta, removed


pkg/storage/mvcc.go line 5294 at r1 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

Let's surface inline values too, rather than error on them.

Isn't that what this is doing? What is this comment asking for?

Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r2, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner)


pkg/storage/mvcc.go line 5294 at r1 (raw file):

Previously, ajwerner wrote…

Isn't that what this is doing? What is this comment asking for?

Oh, sorry. We used to have InlinePolicy too, but we now unconditionally error on inline values -- the TBI will skip over inline values most of the time anyway, so this really shouldn't be used with spans that can have inline values at all. May want to call that out on the request.

@ajwerner ajwerner force-pushed the ajwerner/is-span-empty-request branch from c05cca8 to a1ff07b Compare August 9, 2022 13:47
@ajwerner ajwerner marked this pull request as ready for review August 9, 2022 15:40
@ajwerner ajwerner requested a review from a team August 9, 2022 15:40
@ajwerner
Copy link
Contributor Author

ajwerner commented Aug 9, 2022

@erikgrinaker anything else you want on this?

This commit adds a new request, `IsSpanEmptyRequest`, which checks to see if
there is any data in a key span of any kind. It ignore the GC threshold, and
operates across all versions.

Fixes cockroachdb#85726

Release note: None
@ajwerner ajwerner force-pushed the ajwerner/is-span-empty-request branch from a1ff07b to a0ba58e Compare August 9, 2022 17:29
Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah, this looks good. I assume we don't need a version gate because it'll be gated by the broader schema GC version gate?

Reviewed 1 of 1 files at r3, 4 of 4 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner)

@ajwerner
Copy link
Contributor Author

ajwerner commented Aug 9, 2022

TFTR!

bors r=erikgrinaker

Nah, this looks good. I assume we don't need a version gate because it'll be gated by the broader schema GC version gate?

Correct.

@craig
Copy link
Contributor

craig bot commented Aug 9, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Aug 9, 2022

Build succeeded:

@craig craig bot merged commit 6e0b2f2 into cockroachdb:master Aug 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kv: add command to check whether a span is empty
3 participants