Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage: make key/value printing explicitly redactable #70288

Closed
AlexTalks opened this issue Sep 15, 2021 · 1 comment · Fixed by #81495
Closed

storage: make key/value printing explicitly redactable #70288

AlexTalks opened this issue Sep 15, 2021 · 1 comment · Fixed by #81495
Assignees
@nerdroychan
Labels
A-logging In and around the logging infrastructure. A-storage Relating to our storage engine (Pebble) on-disk storage. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) E-quick-win Likely to be a quick win for someone experienced. E-starter Might be suitable for a starter project for new employees or team members. T-storage Storage Team

Comments

@AlexTalks
Copy link
Contributor

AlexTalks commented Sep 15, 2021 •
edited by cockroach-jira-scripts
Loading

Requested by @knz in #70165:

ideally, we'd have FormatBytesAsKey return a redact.RedactableString. ditto for values.

Currently, the roachpb package injects two utility functions into storage for "pretty printing" byte strings as Keys and Values:

cockroach/pkg/roachpb/data.go

Lines 2539 to 2540 in f278e64

enginepb.FormatBytesAsKey = func(k []byte) string { return Key(k).String() }
enginepb.FormatBytesAsValue = func(v []byte) string { return Value{RawBytes: v}.PrettyPrint() }

As part of the log redaction effort, these functions should each return a redact.RedactableString.

Epic: CRDB-6668

Jira issue: CRDB-10015
@AlexTalks AlexTalks added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-logging In and around the logging infrastructure. T-server-and-security DB Server & Security labels Sep 15, 2021
@knz
Copy link
Contributor

knz commented Sep 17, 2021

this is a kv or storage issue, not server

@blathers-crl blathers-crl bot added the T-storage Storage Team label Sep 17, 2021
@knz knz added A-storage Relating to our storage engine (Pebble) on-disk storage. and removed T-server-and-security DB Server & Security labels Sep 17, 2021
@exalate-issue-sync exalate-issue-sync bot changed the title storage: make key/value printing explicitly redactable storage: make key/value printing explicitly redactable Feb 1, 2022
@mwang1026 mwang1026 added E-starter Might be suitable for a starter project for new employees or team members. E-quick-win Likely to be a quick win for someone experienced. labels May 2, 2022
@nerdroychan nerdroychan self-assigned this May 18, 2022
@nerdroychan nerdroychan mentioned this issue May 19, 2022
Merged
craig bot pushed a commit that referenced this issue May 24, 2022
@rafiss
Merge #81495 #81761
2138d80 
81495: storage: make key/value printing explicitly redactable r=itsbilal,knz a=nerdroychan

This commit makes the formatting functions in storage/enginepb
return explicitly redactable strings.

Resolve #70288.

Release note: None

81761: vendor: update to pgconn 1.12.1 r=otan a=rafiss

This includes a fix to error handling during password auth

Release note: None

Co-authored-by: Chen Chen <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
@craig craig bot closed this as completed in ae17c13 May 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nerdroychan nerdroychan

Labels
A-logging In and around the logging infrastructure. A-storage Relating to our storage engine (Pebble) on-disk storage. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) E-quick-win Likely to be a quick win for someone experienced. E-starter Might be suitable for a starter project for new employees or team members. T-storage Storage Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

storage: make key/value printing explicitly redactable
5 participants
@jlinder @knz @AlexTalks @nerdroychan @mwang1026