err_msg: clarify some ambiguous error message

[ZipFast]

What problem does this PR solve?

Keeping improve BR errors and logs #1055

Some error message in BR is ambiguous and misleading.
i.e.

• If tikv has no permission to read from & write to backup storage, error message cannot clarify whether error occur on br side or tikv side. Users may think why I got "permission denied"? I'm running BR as root!
• If user want to restore tables or databases that have not been backup, error should occur.

Clarify the summary info including:

• data-size => total kv size
• size => backup data size(after compressed)

What is changed and how it works?

1. Adjust the backup and restore summary:

• the data-size field has been changed to total-kv-size
• the size field in backup summary has been changed to backup data size(after compressed)
• the size field in restore summary has been changed to restore data size(after decompressed)

2. Some error messages are changed, and will clarify what is going on when error occur.

• Original error message is as following

[2021/05/18 16:37:28.517 +08:00] [ERROR] [push.go:147] ["backup occur unknown error"] [error="Io(Os { code: 13, kind: PermissionDenied...})"] [stack="github.com/pingcap/br/pkg/backup.(*pushDown).pushBackup\n\t/home/zwj/log_improve/br/pkg/backup/push.go:147\ngithub.jparrowsec.cn/pingcap/br/pkg/backup.(*Client).BackupRange\n\t/home/zwj/log_improve/br/pkg/backup/client.go:540\ngithub.jparrowsec.cn/pingcap/br/pkg/backup.(*Client).BackupRanges.func2.1\n\t/home/zwj/log_improve/br/pkg/backup/client.go:476\ngithub.jparrowsec.cn/pingcap/br/pkg/utils.(*WorkerPool).ApplyOnErrorGroup.func1\n\t/home/zwj/log_improve/br/pkg/utils/worker.go:63\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/root/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57"]

• For now, the error message output is as following:

[2021/05/19 15:43:43.305 +08:00] [ERROR] [push.go:175] [error="[BR:KV:ErrKVStorage]tikv storage occur I/O error: I/O permission denied error occurs on TiKV Node(store id: 1; Address: 127.0.0.1:20162)"] ["work around"="please ensure tikv has permission to read from & write to the storage."] [stack="github.com/pingcap/br/pkg/backup.(*pushDown).pushBackup\n\t/home/zwj/log_improve/br/pkg/backup/push.go:175\ngithub.jparrowsec.cn/pingcap/br/pkg/backup.(*Client).BackupRange\n\t/home/zwj/log_improve/br/pkg/backup/client.go:540\ngithub.jparrowsec.cn/pingcap/br/pkg/backup.(*Client).BackupRanges.func2.1\n\t/home/zwj/log_improve/br/pkg/backup/client.go:476\ngithub.jparrowsec.cn/pingcap/br/pkg/utils.(*WorkerPool).ApplyOnErrorGroup.func1\n\t/home/zwj/log_improve/br/pkg/utils/worker.go:63\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/root/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57"]

3. When user want to restore tables or databases that have not been backup, error should occur.

Check List

Tests

• Manual test (add detailed scripts or steps below)

1. If user want to restore a table or database that has not been backup, an error should occur to notice the user
   
   
2. If user restore a empty table, the summary should notice the user there is nothing to restore.
   
3. Summary message have improve, data-size => total-kv-size and size => backup data size(after compressed)
   

• Integration test

Code changes

• Has exported variable/fields change

Release Note

• No realease note

[ZipFast]

/cc @3pointer /cc @YuJuncen

[ti-chi-bot]

@Zwj-coder: GitHub didn't allow me to request PR reviews from the following users: /cc.

Note that only pingcap members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @3pointer /cc @YuJuncen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[YuJuncen]

/lgtm

[YuJuncen]

Eh, by the way, I have an idea about how to handle this, which might make this PR better. (also cc @3pointer, PTAL)

At here:

br/pkg/backup/push.go

Lines 140 to 141 in b1ed365

	log.Error("backup occur unknown error", zap.String("error", errPb.GetMsg()))
	return res, errors.Annotatef(berrors.ErrKVUnknown, "%v", errPb)

The 'unknown error' omits many context, which make it confusing -- users may think this error occurs in the internal of PR. How about add the store into the context? like:

 return res, errors.Annotatef(berrors.ErrKVUnknown, "error occurs in the TiKV Node %s(@%s): %v", 
    store.GetID(), 
    redact.String(store.GetAddress()), // The net address might be sensitive, so we might need to redact them.
    errPb)

After this, I think, users can notice that this error occurs in TiKV, and we can keep the description of ErrKVUnknown untouched, in case of misleading in non-permission relative cases.

NOTEs:

Unfortunately, currently, our pushDown make multi workers and receive response from them by a channel(respCh), which omits the store info:

br/pkg/backup/push.go

Lines 25 to 29 in b1ed365

	type pushDown struct {
	mgr ClientMgr
	respCh chan *backuppb.BackupResponse
	errCh chan error
	}

You may would like to create a type that contains both response and store info, and replace this type to respCh, then send the store info like this:

push.respCh <- RespAndStore{
	Resp:  resp,
	Store: store,
}

... at here:

br/pkg/backup/push.go

Lines 79 to 83 in b1ed365

	func(resp *backuppb.BackupResponse) error {
	// Forward all responses (including error).
	push.respCh <- resp
	return nil
	},

[ZipFast]

Eh, by the way, I have an idea about how to handle this, which might make this PR better. (also cc @3pointer, PTAL)

At here:

br/pkg/backup/push.go

Lines 140 to 141 in b1ed365

	log.Error("backup occur unknown error", zap.String("error", errPb.GetMsg()))
	return res, errors.Annotatef(berrors.ErrKVUnknown, "%v", errPb)

The 'unknown error' omits many context, which make it confusing -- users may think this error occurs in the internal of PR. How about add the store into the context? like:

 return res, errors.Annotatef(berrors.ErrKVUnknown, "error occurs in the TiKV Node %s(@%s): %v", 
    store.GetID(), 
    redact.String(store.GetAddress()), // The net address might be sensitive, so we might need to redact them.
    errPb)

I think after this, users can notice that this error occurs in TiKV, and we can keep the description of ErrKVUnknown untouched, in case of misleading in non-permission relative cases.

NOTEs:

Unfortunately, currently, our pushDown make multi workers and receive response from them by a channel(respCh), which omits the store info:

br/pkg/backup/push.go

Lines 25 to 29 in b1ed365

	type pushDown struct {
	mgr ClientMgr
	respCh chan *backuppb.BackupResponse
	errCh chan error
	}

You may would like to create a type that contains both response and store info, and replace this type to respCh, then send the store info like this:

push.respCh <- RespAndStore{
	Resp:  resp,
	Store: store,
}

... at here:

br/pkg/backup/push.go

Lines 79 to 83 in b1ed365

	func(resp *backuppb.BackupResponse) error {
	// Forward all responses (including error).
	push.respCh <- resp
	return nil
	},

I think since the error stack contains both error message from tikv and br, if the error occur at the tikv side, it's tikv's responsibility to report detailed error message.

[YuJuncen]

@Zwj-coder Yep, but I think BR can figure out on WHICH TiKV error happens , IMO this could be helpful for debugging. 🤔

[ZipFast]

@YuJuncen I agree, but if the tikv cluster is configured in the same way, one of them have permission problem probably means rest of them also have same problem, so we could add this store help information for debugging usage.

[YuJuncen]

@Zwj-coder Well, that is true. Even I'm still afraid this message being misleading in some conditions... 😂

[ZipFast]

/cc @YuJuncen @3pointer

[3pointer]

/lgtm

BTW Is internal error better than unknown error here?🤔️

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• 3pointer
• YuJuncen

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

[3pointer]

/run-integration-tests

[3pointer]

we could have the same tables in different dbs.
It's better to change the key from tbl to db.tbl.

[IANTHEREAL]

/lgtm

[ti-chi-bot]

In response to a cherrypick label: new pull request created: #1131.

[ti-chi-bot]

In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/br#release-4.0 from head ti-chi-bot:cherry-pick-1076-to-release-4.0: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"PullRequest","code":"custom","message":"A pull request already exists for ti-chi-bot:cherry-pick-1076-to-release-4.0."}],"documentation_url":"https://docs.github.com/rest/reference/pulls#create-a-pull-request"}

[ti-chi-bot]

In response to a cherrypick label: new pull request created: #1132.

[ti-chi-bot]

In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/br#release-5.0 from head ti-chi-bot:cherry-pick-1076-to-release-5.0: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"PullRequest","code":"custom","message":"A pull request already exists for ti-chi-bot:cherry-pick-1076-to-release-5.0."}],"documentation_url":"https://docs.github.com/rest/reference/pulls#create-a-pull-request"}