Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: include range key info in ScanStats #93326

Closed
erikgrinaker opened this issue Dec 9, 2022 · 2 comments · Fixed by #94345
Closed

storage: include range key info in ScanStats #93326

erikgrinaker opened this issue Dec 9, 2022 · 2 comments · Fixed by #94345
Assignees
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-storage Storage Team

Comments

@erikgrinaker
Copy link
Contributor

erikgrinaker commented Dec 9, 2022

The roachpb.ScanStats struct contains scan statistics that are attached to BatchResponses for observability purposes. This should include information about range keys as well, which will require corresponding changes to pebble.IteratorStats.

cockroach/pkg/storage/mvcc.go

Lines 3556 to 3559 in 19f2fad

KeyBytes: stats.InternalStats.KeyBytes,
ValueBytes: stats.InternalStats.ValueBytes,
PointCount: stats.InternalStats.PointCount,
PointsCoveredByRangeTombstones: stats.InternalStats.PointsCoveredByRangeTombstones,

Jira issue: CRDB-22275

@erikgrinaker erikgrinaker added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-storage Storage Team labels Dec 9, 2022
@blathers-crl blathers-crl bot added the A-storage Relating to our storage engine (Pebble) on-disk storage. label Dec 9, 2022
@erikgrinaker
Copy link
Contributor Author

@jbowens Would this be quick to wire up? Would have been useful to debug a kvnemesis test failure over in #93312.

@jbowens
Copy link
Collaborator

jbowens commented Dec 9, 2022

I have a PR (cockroachdb/pebble#1871) that's been languishing. I'll try to get it across the finish line next week.

@jbowens jbowens self-assigned this Dec 20, 2022
craig bot pushed a commit that referenced this issue Jan 3, 2023
94010: sql/importer: unskip TestCSVImportCanBeResumed r=rafiss a=stevendanna

This unskips the test and adds a memory limit. The test completed 1500+ runs under stress. I believe #93782 resolved the main source of flakiness here.

Fixes #91828

Release note: None

94345: storage: include range key stats in ScanStats r=erikgrinaker a=jbowens

Include the new, low-level Pebble range key iterator stats (introduced in cockroachdb/pebble#1871) in roachpb.ScanStats.

Informs #77580.
Close #93326.

Epic: None
Release note: None

94458: logictest: fix flakes from mixed version testing r=ZhouXing19 a=rafiss

fixes #92637

The main fix was to wait for each node to be reachable before beginning the upgrade process. This also includes a version bump that has better logging to make it easier to debug.

Release note: None

Co-authored-by: Steven Danna <[email protected]>
Co-authored-by: Jackson Owens <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
@craig craig bot closed this as completed in e001bf9 Jan 3, 2023
blathers-crl bot pushed a commit that referenced this issue Jan 3, 2023
Include the new, low-level Pebble range key iterator stats (introduced in
cockroachdb/pebble#1871) in roachpb.ScanStats.

Informs #77580.
Close #93326.

Epic: None
Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-storage Storage Team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants