Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: support generating leading and hash_join_build hint from physical plan #55195

Merged
merged 34 commits into from
Aug 9, 2024

Conversation

time-and-fate
Copy link
Member

@time-and-fate time-and-fate commented Aug 5, 2024

What problem does this PR solve?

Issue Number: close #55280

What changed and how does it work?

As the title says.

The overall structure of methods in hint_utils.go:

  1. Methods for join method hints: genJoinMethodHintForSinglePhysicalJoin()
  2. Methods for join order hints: genJoinOrderHintFromRootPhysicalJoin(), extractOrderedPhysicalJoinGroup()
  3. Methods for extracting table names and QB names, which are common methods for both join method hints and join order hints: genHintTblForJoinNodes(), genHintTblForSingleJoinNode(), extractTableAsName()

Please see the comments for details since the changes involve many implementation details.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. sig/planner SIG: Planner labels Aug 5, 2024
Copy link

codecov bot commented Aug 5, 2024

Codecov Report

Attention: Patch coverage is 87.30159% with 32 lines in your changes missing coverage. Please review.

Project coverage is 74.2622%. Comparing base (240702e) to head (98008f8).
Report is 1 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #55195        +/-   ##
================================================
+ Coverage   72.8938%   74.2622%   +1.3683%     
================================================
  Files          1571       1571                
  Lines        439731     440116       +385     
================================================
+ Hits         320537     326840      +6303     
+ Misses        99471      93056      -6415     
- Partials      19723      20220       +497     
Flag Coverage Δ
integration 27.8698% <0.3968%> (?)
unit 71.8153% <87.3015%> (-0.1148%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.9567% <ø> (ø)
parser ∅ <ø> (∅)
br 52.5020% <ø> (+6.6232%) ⬆️

@time-and-fate
Copy link
Member Author

/retest

@time-and-fate
Copy link
Member Author

/retest

@time-and-fate
Copy link
Member Author

CI pass now

@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 5, 2024
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 7, 2024
@time-and-fate
Copy link
Member Author

/retest

@time-and-fate time-and-fate changed the title planner: [WIP] planner: support generating leading and hash_join_build hint from physical plan Aug 7, 2024
func genJoinMethodHintForSinglePhysicalJoin(
sctx base.PlanContext,
joinType string,
parentOffset int,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we rename parentOffset to parentQBOffset or parentQueryBlockOffset? Offset seems unclear, better to add keyword QB or QueryBlock into it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

}
guessQBOffset = false
var dbName, tableName *model.CIStr
if qbOffset != parentOffset {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can only happen when the joined node is from a sub-query? Better to add some comments.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is not changed. I move the previous comments here.

ht *ast.HintTable,
) {
qbOffset = joinNode.QueryBlockOffset()
selfOffset := joinNode.QueryBlockOffset()
qbOffset = selfOffset
if qbOffset == -1 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this can be -1? The optimizer forgot to set this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Sometimes join reorder process won't maintain it and set it to -1.

if qbOffset < 0 || ht == nil {
qbOffsets = append(qbOffsets, -1)
hintTbls = append(hintTbls, nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we "continue" after these 2 append operations?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Updated.

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Aug 8, 2024
@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 8, 2024
@time-and-fate
Copy link
Member Author

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 9, 2024
@ti-chi-bot ti-chi-bot bot added approved lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Aug 9, 2024
Copy link

ti-chi-bot bot commented Aug 9, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-08-08 07:49:29.215410644 +0000 UTC m=+511099.082509732: ☑️ agreed by qw4990.
  • 2024-08-09 09:24:03.647082574 +0000 UTC m=+603173.514181663: ☑️ agreed by tangenta.

Copy link

ti-chi-bot bot commented Aug 9, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: qw4990, tangenta, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@time-and-fate
Copy link
Member Author

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 9, 2024
@ti-chi-bot ti-chi-bot bot merged commit 6df7aba into pingcap:master Aug 9, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support generating leading and hash_join_build hint for quick binding
4 participants