Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support rewriting subquery expressions in simplify_expressions #3770

Closed
Tracked by #4763
andygrove opened this issue Oct 8, 2022 · 1 comment · Fixed by #13066
Closed
Tracked by #4763

Support rewriting subquery expressions in simplify_expressions #3770

andygrove opened this issue Oct 8, 2022 · 1 comment · Fixed by #13066
Labels
enhancement New feature or request optimizer Optimizer rules

Comments

@andygrove
Copy link
Member

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
PR #3764 reverts a regression in optimizing subqueries but does so by running simplify_expressions a second time, after subqueries have been converted to joins.

This is not the ideal solution.

Describe the solution you'd like
simplify_expressions should rewrite subqueries instead. This will support all subqueries, not just the ones that can be converted to joins.

Describe alternatives you've considered
None

Additional context
See conversation #3764 (comment)
@andygrove andygrove added enhancement New feature or request optimizer Optimizer rules labels Oct 8, 2022
@andygrove andygrove mentioned this issue Oct 8, 2022
Merged
@alamb alamb mentioned this issue Oct 11, 2022
Open
@HaoYang670
Copy link
Contributor

Do we need to consider the schema infomation of the parent query when optimizing the subquery?

@devinjdangelo devinjdangelo mentioned this issue Mar 9, 2024
Closed
@eejbyfeldt eejbyfeldt mentioned this issue Oct 21, 2024
Closed
eejbyfeldt added a commit to eejbyfeldt/datafusion that referenced this issue Oct 22, 2024
@eejbyfeldt
Run optimzer rules on subqueries by default
bf8b752 
This patch makes it so that rules the configure an `apply_order` will
also include subqueries in their traversel.

This is a step twoards being able to run TPC-DS q41 (apache#4763) which has
an expressions that needs simplification before we can decorrelate the
subquery.

This closes apache#3770 and maybe apache#2480
@eejbyfeldt eejbyfeldt mentioned this issue Oct 22, 2024
Merged
eejbyfeldt added a commit to eejbyfeldt/datafusion that referenced this issue Oct 22, 2024
@eejbyfeldt
Run optimzer rules on subqueries by default
df84a84 
This patch makes it so that rules the configure an `apply_order` will
also include subqueries in their traversel.

This is a step twoards being able to run TPC-DS q41 (apache#4763) which has
an expressions that needs simplification before we can decorrelate the
subquery.

This closes apache#3770 and maybe apache#2480
@Dandandan Dandandan mentioned this issue Oct 23, 2024
Open
4 tasks
Dandandan pushed a commit that referenced this issue Oct 23, 2024
@eejbyfeldt
Run optimzer rules on subqueries by default (#13066)
3aa9714 
This patch makes it so that rules the configure an `apply_order` will
also include subqueries in their traversel.

This is a step twoards being able to run TPC-DS q41 (#4763) which has
an expressions that needs simplification before we can decorrelate the
subquery.

This closes #3770 and maybe #2480
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request optimizer Optimizer rules
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Run (logical) optimizers on subqueries eejbyfeldt/datafusion
2 participants
@andygrove @HaoYang670