Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statistics: handle exchange partition event #49043

Merged

Conversation

Rustin170506
Copy link
Member

What problem does this PR solve?

Issue Number: close #47354

Problem Summary:

What changed and how does it work?

Based on this design doc, we will only update the count and modfiy_count for it. Then it will use the count and modify_count to determine if we need to trigger an auto-analysis task.

So in this PR, I updated the global stats with the old partition info and table info.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 30, 2023
Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check

@Rustin170506 Rustin170506 marked this pull request as draft November 30, 2023 08:24
@ti-chi-bot ti-chi-bot bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 30, 2023
@Rustin170506 Rustin170506 marked this pull request as ready for review November 30, 2023 09:34
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 30, 2023
Copy link

codecov bot commented Nov 30, 2023

Codecov Report

Merging #49043 (2f6c2c3) into master (373608f) will increase coverage by 0.9592%.
Report is 15 commits behind head on master.
The diff coverage is 5.8823%.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #49043        +/-   ##
================================================
+ Coverage   71.0418%   72.0011%   +0.9592%     
================================================
  Files          1368       1407        +39     
  Lines        402868     414727     +11859     
================================================
+ Hits         286205     298608     +12403     
- Misses        96722      97275       +553     
+ Partials      19941      18844      -1097     
Flag Coverage Δ
integration 43.6727% <5.8823%> (?)
unit 71.0422% <ø> (+0.0003%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.9663% <ø> (ø)
parser ∅ <ø> (∅)
br 48.0817% <ø> (-4.8869%) ⬇️

@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 30, 2023
Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check

@Rustin170506
Copy link
Member Author

/retest

Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check

@Rustin170506 Rustin170506 requested a review from zimulala December 4, 2023 05:20
Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check

Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check

Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: hi-rustin <[email protected]>
@Rustin170506 Rustin170506 force-pushed the rustin-patch-exchange-parittion branch from ad90dd2 to 2f6c2c3 Compare December 7, 2023 08:45
// updateStatsWithCountDeltaAndModifyCountDelta updates
// the global stats with the given count delta and modify count delta.
// Only used by some special DDLs, such as exchange partition.
func updateStatsWithCountDeltaAndModifyCountDelta(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TiDB supports changing multiple schemas in ALTER TABLE. so it can exchange table partitions for multi-table. Do you need it to support batch update/insert here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You cannot exchange more than one partition in TiDB. Therefore, it's okay. "Even if you exchange them together, they are still considered as different events."

@@ -2594,6 +2594,9 @@ func (w *worker) onExchangeTablePartition(d *ddlCtx, t *meta.Meta, job *model.Jo

// exchange table meta id
pt.ExchangePartitionInfo = nil
// Used below to update the partitioned table's stats meta.
originalPartitionDef := partDef.Clone()
originalNt := nt.Clone()
partDef.ID, nt.ID = nt.ID, partDef.ID
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc: @hawkingrei As you can see, we only exchange one partition here.

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Dec 7, 2023
Copy link
Member Author

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check

Copy link

ti-chi-bot bot commented Dec 8, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: time-and-fate, zimulala

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm approved and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 8, 2023
Copy link

ti-chi-bot bot commented Dec 8, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-12-07 11:16:52.549588461 +0000 UTC m=+1699041.214814658: ☑️ agreed by time-and-fate.
  • 2023-12-08 07:59:11.088163542 +0000 UTC m=+1773579.753389722: ☑️ agreed by zimulala.

@ti-chi-bot ti-chi-bot bot merged commit cc6bf2b into pingcap:master Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved component/statistics lgtm release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TiDB can not update stats correctly after exchange a partition
4 participants