planner: Mpp outer join build side (#25130) #25142

ti-srebot · 2021-06-04T06:18:20Z

cherry-pick #25130 to release-5.0
You can switch your code base to this Pull Request by using git-extras:

# In tidb repo:
git pr https://github.com/pingcap/tidb/pull/25142

After apply modifications, you can push your change to this PR via:

git push [email protected]:ti-srebot/tidb.git pr/25142:release-5.0-c59b3bcea6ef

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:
In MPP outer join, it always use the inner table as the build table, however, TiFlash actually support use outer table as the build table in outer join, so we can enable this to speed up query.

What is changed and how it works?

Proposal: xxx

What's Changed:

How it Works:
for mpp outer join, choose the build table based on the row count.

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List

Tests

Integration test
Manual test (add detailed scripts or steps below)

Side effects

Performance regression
- Consumes more CPU
- Consumes more MEM
Breaking backward compatibility

Release note

allow mpp outer join to choose the build table based on table row count

Signed-off-by: ti-srebot <[email protected]>

ti-srebot · 2021-06-04T06:18:22Z

/run-all-tests

ti-srebot · 2021-06-04T06:18:27Z

@windtalker you're already a collaborator in bot's repo.

This reverts commit 7d777df.

…lease-5.0-c59b3bcea6ef

…25240)

windtalker · 2021-06-18T01:56:25Z

Cherry-pick #25240 into this pr.

fzhedu

add more cases?

fzhedu · 2021-06-18T02:15:22Z

executor/tiflash_test.go

@@ -129,6 +132,9 @@ func (s *tiflashTestSuite) TestReadUnsigedPK(c *C) {
 	tk.MustExec("set @@session.tidb_isolation_read_engines=\"tiflash\"")
 	tk.MustExec("set @@session.tidb_allow_mpp=ON")


enforce_mpp?

enforce_mpp is not supported in release-5.0

fzhedu · 2021-06-18T02:17:59Z

planner/core/exhaust_physical_plans.go

+		if useBCJ || p.ctx.GetSessionVars().MPPOuterJoinFixedBuildSide || len(p.OtherConditions) > 0 {
+			if p.JoinType == LeftOuterJoin {
+				preferredBuildIndex = 1
+			}
+		} else if p.children[0].statsInfo().Count() > p.children[1].statsInfo().Count() {
+			preferredBuildIndex = 1
+		}
+	}


does tests cover these cases?

fzhedu · 2021-06-18T02:27:04Z

planner/core/testdata/integration_serial_suite_in.json

@@ -38,6 +38,27 @@
      "explain format = 'brief' select count(*) from fact_t where d1_k not in (select d1_k from d1_t)"
    ]
  },
+  {
+    "name": "TestMPPOuterJoinBuildSideForBroadcastJoin",
+    "cases": [


add more case for other conditions, cross join?

also need randgen tests to guarantee rightness.

If the outer join has other condition, it will always use inner table as the build side, this behavior is not changed by this pr. For cross join, TiFlash will convert Cross right join to Cross left join, so is not affected by this pr.

fzhedu

/LGTM

ti-chi-bot · 2021-06-18T02:31:29Z

@fzhedu: Please use GitHub review feature instead of /lgtm [cancel] when you want to submit review to the pull request.
For how to use GitHub review feature, see also this document provided by GitHub.

For the reason we drop support to the commands, see also this page.
This reply is being used as a temporary reply during the migration of review process and will be removed on July 1st.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2021-06-18T05:06:59Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

fzhedu
hanfei1991

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

windtalker · 2021-06-18T06:20:13Z

/merge

ti-chi-bot · 2021-06-18T06:20:15Z

@windtalker: /merge is only allowed for the committers, you can assign this pull request to the committer in list by filling /assign @committer in the comment to help merge this pull request.

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

xiongjiwei · 2021-06-18T06:20:56Z

/merge

ti-chi-bot · 2021-06-18T06:20:59Z

This pull request has been accepted and is ready to merge.

Commit hash: 0d88cc6

windtalker · 2021-06-18T06:25:34Z

/run-all-tests

cherry pick pingcap#25130 to release-5.0

7d777df

Signed-off-by: ti-srebot <[email protected]>

ti-srebot requested a review from a team as a code owner June 4, 2021 06:18

ti-srebot requested review from hanfei1991 and removed request for a team June 4, 2021 06:18

ti-chi-bot added the do-not-merge/cherry-pick-not-approved label Jun 4, 2021

ti-srebot mentioned this pull request Jun 4, 2021

planner: Mpp outer join build side #25130

Merged

ti-srebot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. type/5.0-cherry-pick labels Jun 4, 2021

ti-srebot requested review from fzhedu and time-and-fate June 4, 2021 06:18

ti-srebot added this to the v5.0.2 milestone Jun 4, 2021

ti-srebot assigned windtalker Jun 4, 2021

windtalker added 2 commits June 18, 2021 09:40

Revert "cherry pick pingcap#25130 to release-5.0"

47a1322

This reverts commit 7d777df.

Merge branch 'release-5.0' of https://github.com/pingcap/tidb into re…

8e6e58b

…lease-5.0-c59b3bcea6ef

ti-chi-bot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 18, 2021

windtalker added 2 commits June 18, 2021 09:50

planner: Mpp outer join build side (pingcap#25130)

761e53c

planner: disable mpp_outer_join_fixed_build_side by default (pingcap#…

6bd729d

…25240)

ti-srebot requested a review from a team as a code owner June 18, 2021 01:55

ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 18, 2021

github-actions bot added sig/execution SIG execution sig/sql-infra SIG: SQL Infra labels Jun 18, 2021

refine

0d88cc6

fzhedu reviewed Jun 18, 2021

View reviewed changes

fzhedu approved these changes Jun 18, 2021

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 18, 2021

hanfei1991 approved these changes Jun 18, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 18, 2021

xiongjiwei approved these changes Jun 18, 2021

View reviewed changes

zhouqiang-cl added backport-5.0.3 cherry-pick-approved Cherry pick PR approved by release team. labels Jun 18, 2021

ti-chi-bot removed the do-not-merge/cherry-pick-not-approved label Jun 18, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 18, 2021

Merge branch 'release-5.0' into release-5.0-c59b3bcea6ef

4ed6f9f

ti-chi-bot merged commit fbe99ff into pingcap:release-5.0 Jun 18, 2021

zhouqiang-cl modified the milestones: v5.0.2, v5.0.3 Jun 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner: Mpp outer join build side (#25130) #25142

planner: Mpp outer join build side (#25130) #25142

ti-srebot commented Jun 4, 2021 •

edited

Loading

ti-srebot commented Jun 4, 2021

ti-srebot commented Jun 4, 2021

windtalker commented Jun 18, 2021 •

edited

Loading

fzhedu left a comment

fzhedu Jun 18, 2021

windtalker Jun 18, 2021

fzhedu Jun 18, 2021

windtalker Jun 18, 2021

fzhedu Jun 18, 2021

windtalker Jun 18, 2021

fzhedu left a comment

ti-chi-bot commented Jun 18, 2021

ti-chi-bot commented Jun 18, 2021

windtalker commented Jun 18, 2021

ti-chi-bot commented Jun 18, 2021

xiongjiwei commented Jun 18, 2021

ti-chi-bot commented Jun 18, 2021

windtalker commented Jun 18, 2021

		@@ -129,6 +132,9 @@ func (s tiflashTestSuite) TestReadUnsigedPK(c C) {
		tk.MustExec("set @@session.tidb_isolation_read_engines=\"tiflash\"")
		tk.MustExec("set @@session.tidb_allow_mpp=ON")

planner: Mpp outer join build side (#25130) #25142

planner: Mpp outer join build side (#25130) #25142

Conversation

ti-srebot commented Jun 4, 2021 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Related changes

Check List

Release note

ti-srebot commented Jun 4, 2021

ti-srebot commented Jun 4, 2021

windtalker commented Jun 18, 2021 • edited Loading

fzhedu left a comment

Choose a reason for hiding this comment

fzhedu Jun 18, 2021

Choose a reason for hiding this comment

windtalker Jun 18, 2021

Choose a reason for hiding this comment

fzhedu Jun 18, 2021

Choose a reason for hiding this comment

windtalker Jun 18, 2021

Choose a reason for hiding this comment

fzhedu Jun 18, 2021

Choose a reason for hiding this comment

windtalker Jun 18, 2021

Choose a reason for hiding this comment

fzhedu left a comment

Choose a reason for hiding this comment

ti-chi-bot commented Jun 18, 2021

ti-chi-bot commented Jun 18, 2021

windtalker commented Jun 18, 2021

ti-chi-bot commented Jun 18, 2021

xiongjiwei commented Jun 18, 2021

ti-chi-bot commented Jun 18, 2021

windtalker commented Jun 18, 2021

ti-srebot commented Jun 4, 2021 •

edited

Loading

windtalker commented Jun 18, 2021 •

edited

Loading