Add latency formula #9763

you06 · 2022-07-28T16:15:51Z

First-time contributors' checklist

I've signed Contributor License Agreement that's required for repo owners to accept my contribution.

What is changed, added or deleted? (Required)

Add a document about query latency breakdown.

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

What is the related PR or file link(s)?

This PR is translated from:
Other reference link(s):

Do your changes match any of the following descriptions?

Delete files
Change aliases
Need modification after applied to another branch
Might cause conflicts after applied to another branch

Signed-off-by: you06 <[email protected]>

ti-chi-bot · 2022-07-28T16:15:53Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

Oreoxmt
ran-huang

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: you06 <[email protected]>

you06 · 2022-08-03T06:49:50Z

@cfzjywxk @sticnarf can you give some advice on this document?

tidb-latency-formula.md

Signed-off-by: you06 <[email protected]>

you06 · 2022-08-05T08:06:27Z

I added the railroad diagrams and here is a screenshot of doc page.

@cfzjywxk @sticnarf PTAL again.

Signed-off-by: you06 <[email protected]>

latency-breakdown.md

sticnarf · 2022-08-09T06:53:16Z

latency-breakdown.md

+prewrite_ratio = ceil(
+    sum(rate(tidb_tikvclient_txn_regions_num_sum{type="2pc_prewrite"})) /
+    sum(rate(tidb_tikvclient_txn_regions_num_count{type="2pc_prewrite"})) /
+    128


This should be explained (committer-concurrency)

sticnarf · 2022-08-09T06:56:14Z

latency-breakdown.md

+)
+```
+
+TiKV supports Async IO Raft since v5.3.0, which changes the process of commit.


Let's clarify it's not enabled by default. It's enabled when store-io-pool-size > 0

Signed-off-by: you06 <[email protected]>

latency-breakdown.md

Oreoxmt · 2022-09-09T02:47:31Z

latency-breakdown.md

@@ -683,7 +763,7 @@ async io enabled commit = max(
 )
 ```

-TiKV supports Async IO Raft since v5.3.0(only enabled when [`store-io-pool-size > 0`](/tikv-configuration-file.md#store-io-pool-size-new-in-v530)), which changes the process of commit.
+Since v5.3.0, TiKV supports Async IO Raft, which only enabled when setting the [`store-io-pool-size`](/tikv-configuration-file.md#store-io-pool-size-new-in-v530) as a positive value, which changes the process of commit. The `persist log locally duration` and `wait by write worker duration` are calculated as:


What is Async IO? "Async IO" is not introduced in the store-io-pool-size configuration item.

"Async IO" means handle writes of committing Raft log by a "StoreWriter thread pool", I'll change it.

Because "Async IO" is used frequently between developers, I kept it and added a description.

latency-breakdown.md

Oreoxmt · 2022-09-09T03:44:07Z

@you06 @ran-huang Preview link is here

Signed-off-by: you06 <[email protected]>

ran-huang · 2022-09-14T02:20:06Z

latency-breakdown.md

+
+# Latency Breakdown
+
+This document breaks down the latency into metrics and then analyzes it from the user's perspective, which provides you with a deep insight into time cost during TiDB SQL queries. This is a guide to TiDB's critical path diagnosis.


Can you briefly define "latency"? What does it mean in this doc?

Give your readers a bird's-eye view of the whole doc. For example:

This document analyzes TiDB latency from xx aspects: - General SQL layer - Read queries - ...

latency-breakdown.md

ran-huang · 2022-09-14T03:12:21Z

latency-breakdown.md

+
+## Read queries
+
+Read queries have only a single process form.


I don't understand "single process form". Can you explain it?

What I mean is there is only one form for processing read requests, and there are no extra works after the read process.

latency-breakdown.md

Signed-off-by: you06 <[email protected]>

sticnarf

LGTM

Oreoxmt · 2022-09-19T05:55:46Z

/merge

ti-chi-bot · 2022-09-19T05:55:48Z

@Oreoxmt: /merge in this pull request requires 2 approval(s).

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Oreoxmt

LGTM

Oreoxmt · 2022-09-19T05:56:34Z

/merge

ti-chi-bot · 2022-09-19T05:56:36Z

This pull request has been accepted and is ready to merge.

Commit hash: 321d6f6

you06 added 2 commits July 28, 2022 08:47

wip

5a57372

Signed-off-by: you06 <[email protected]>

write done the first version

812d076

Signed-off-by: you06 <[email protected]>

ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 28, 2022

ti-chi-bot requested a review from shichun-0415 July 28, 2022 16:15

ti-chi-bot added missing-translation-status This PR does not have translation status info. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 28, 2022

shichun-0415 added translation/doing This PR's assignee is translating this PR. and removed missing-translation-status This PR does not have translation status info. labels Jul 30, 2022

update

ef12a5c

Signed-off-by: you06 <[email protected]>

ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 3, 2022

fix lint

9524c10

Signed-off-by: you06 <[email protected]>

you06 marked this pull request as ready for review August 3, 2022 06:40

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 3, 2022

remove tailing lines

7ccc77c

Signed-off-by: you06 <[email protected]>

cfzjywxk reviewed Aug 3, 2022

View reviewed changes

tidb-latency-formula.md Outdated Show resolved Hide resolved

tidb-latency-formula.md Outdated Show resolved Hide resolved

sticnarf reviewed Aug 3, 2022

View reviewed changes

you06 added 2 commits August 4, 2022 22:52

address comment

3630d8e

Signed-off-by: you06 <[email protected]>

updates

77fa198

Signed-off-by: you06 <[email protected]>

you06 added 2 commits August 5, 2022 17:31

fix dead link

091819e

Signed-off-by: you06 <[email protected]>

remove manual line break

9fafba8

Signed-off-by: you06 <[email protected]>

you06 requested review from cfzjywxk and sticnarf August 9, 2022 02:55

sticnarf reviewed Aug 9, 2022

View reviewed changes

address comment

0b4e2f1

Signed-off-by: you06 <[email protected]>

sticnarf approved these changes Aug 11, 2022

View reviewed changes

fix lint

f4f8fa7

Signed-off-by: you06 <[email protected]>

Oreoxmt reviewed Sep 9, 2022

View reviewed changes

review: fix format and update description

6285301

Oreoxmt force-pushed the add-latency-formula branch from 82d1522 to 6285301 Compare September 9, 2022 03:11

Oreoxmt requested a review from ran-huang September 9, 2022 03:12

address comment

6c35ccd

Signed-off-by: you06 <[email protected]>

Oreoxmt self-requested a review September 13, 2022 08:38

ran-huang reviewed Sep 14, 2022

View reviewed changes

apply suggestions from@ran-huang

70fe411

Oreoxmt mentioned this pull request Sep 15, 2022

Add latency formula pingcap/docs-cn#11308

Merged

13 tasks

add glossary

4d9c476

Signed-off-by: you06 <[email protected]>

ran-huang approved these changes Sep 15, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 15, 2022

Oreoxmt added translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR. and removed translation/doing This PR's assignee is translating this PR. labels Sep 15, 2022

Oreoxmt added 2 commits September 15, 2022 17:29

Merge branch 'master' into add-latency-formula

8d48838

review

8caac47

sticnarf approved these changes Sep 16, 2022

View reviewed changes

Merge branch 'master' into add-latency-formula

321d6f6

Oreoxmt approved these changes Sep 19, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Sep 19, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 19, 2022

Oreoxmt closed this Sep 19, 2022

Oreoxmt reopened this Sep 19, 2022

ti-chi-bot merged commit 2e8f2aa into pingcap:master Sep 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add latency formula #9763

Add latency formula #9763

you06 commented Jul 28, 2022

ti-chi-bot commented Jul 28, 2022 •

edited

Loading

you06 commented Aug 3, 2022

you06 commented Aug 5, 2022

sticnarf Aug 9, 2022

sticnarf Aug 9, 2022

Oreoxmt Sep 9, 2022

you06 Sep 9, 2022

you06 Sep 9, 2022

Oreoxmt commented Sep 9, 2022 •

edited

Loading

ran-huang Sep 14, 2022

ran-huang Sep 14, 2022

you06 Sep 15, 2022

sticnarf left a comment

Oreoxmt commented Sep 19, 2022

ti-chi-bot commented Sep 19, 2022

Oreoxmt left a comment

Oreoxmt commented Sep 19, 2022

ti-chi-bot commented Sep 19, 2022


		# Latency Breakdown

		This document breaks down the latency into metrics and then analyzes it from the user's perspective, which provides you with a deep insight into time cost during TiDB SQL queries. This is a guide to TiDB's critical path diagnosis.


		## Read queries

		Read queries have only a single process form.

Add latency formula #9763

Add latency formula #9763

Conversation

you06 commented Jul 28, 2022

First-time contributors' checklist

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

What is the related PR or file link(s)?

Do your changes match any of the following descriptions?

ti-chi-bot commented Jul 28, 2022 • edited Loading

you06 commented Aug 3, 2022

you06 commented Aug 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Oreoxmt commented Sep 9, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sticnarf left a comment

Choose a reason for hiding this comment

Oreoxmt commented Sep 19, 2022

ti-chi-bot commented Sep 19, 2022

Oreoxmt left a comment

Choose a reason for hiding this comment

Oreoxmt commented Sep 19, 2022

ti-chi-bot commented Sep 19, 2022

ti-chi-bot commented Jul 28, 2022 •

edited

Loading

Oreoxmt commented Sep 9, 2022 •

edited

Loading