Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] (All) logical optimizers should optimize subqueries #2480

Open
jon-chuang opened this issue May 7, 2022 · 0 comments
Open

[Feature] (All) logical optimizers should optimize subqueries #2480

jon-chuang opened this issue May 7, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@jon-chuang
Copy link

Describe the solution you'd like
Logical optimizer only walks LogicalPlan node inputs. However, realistically, every optimizer should now also walk every subquery by matching on Filter(Filter { expr: conjunction(vec![.., InSubquery/Exists { subquery }, ..] } and optimize the subquery too.

One solution may be to declare subqueries as one of the inputs to the plan nodes...? However, this may "denormalize" certain things, so it may be better to add a helper function (based on e.g. visit/accept) that makes it easy for all optimizations to detect and walk subqueries.
@jon-chuang jon-chuang added the enhancement New feature or request label May 7, 2022
@jon-chuang jon-chuang changed the title [Feature] Logical optimizer should walk subqueries [Feature] (All) logical optimizers should optimize subqueries May 7, 2022
@jon-chuang jon-chuang mentioned this issue May 7, 2022
Closed
15 tasks
@alamb alamb mentioned this issue Mar 5, 2023
Open
13 tasks
eejbyfeldt added a commit to eejbyfeldt/datafusion that referenced this issue Oct 22, 2024
@eejbyfeldt
Run optimzer rules on subqueries by default
bf8b752 
This patch makes it so that rules the configure an `apply_order` will
also include subqueries in their traversel.

This is a step twoards being able to run TPC-DS q41 (apache#4763) which has
an expressions that needs simplification before we can decorrelate the
subquery.

This closes apache#3770 and maybe apache#2480
@eejbyfeldt eejbyfeldt mentioned this issue Oct 22, 2024
Merged
eejbyfeldt added a commit to eejbyfeldt/datafusion that referenced this issue Oct 22, 2024
@eejbyfeldt
Run optimzer rules on subqueries by default
df84a84 
This patch makes it so that rules the configure an `apply_order` will
also include subqueries in their traversel.

This is a step twoards being able to run TPC-DS q41 (apache#4763) which has
an expressions that needs simplification before we can decorrelate the
subquery.

This closes apache#3770 and maybe apache#2480
Dandandan pushed a commit that referenced this issue Oct 23, 2024
@eejbyfeldt
Run optimzer rules on subqueries by default (#13066)
3aa9714 
This patch makes it so that rules the configure an `apply_order` will
also include subqueries in their traversel.

This is a step twoards being able to run TPC-DS q41 (#4763) which has
an expressions that needs simplification before we can decorrelate the
subquery.

This closes #3770 and maybe #2480
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jon-chuang