-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move LogicalPlan
enum to datafusion-expr
crate
#2294
Conversation
AMD64 test failure seems unrelated. Maybe an unreliable test?
|
@@ -0,0 +1,23 @@ | |||
// Licensed to the Apache Software Foundation (ASF) under one |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if making this a separate crate like datafusion-logical-plan
might be considered? Is there any reason the logical plan structs need to be in the expr? I could (potentially) see a use of just Expr
s without also LogicalPlan
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was working towards having the logical plan and logical expressions in the same create (currently named datafusion-expr
but potentially could be renamed to datafusion-logical-plan
).
The main motivation is that I was planning on adding something like:
pub enum Expr {
Exists {
subquery: Arc<LogicalPlan>,
negated: bool
}
...
}
This would be similar to Apache Spark:
case class Subquery(child: LogicalPlan, correlated: Boolean) extends OrderPreservingUnaryNode {
Also similar to Calcite:
public class RexSubQuery extends RexCall {
public final RelNode rel;
I'm not sure how else we would model this but am open to suggestions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jackwener @xudong963 @mingmwang @paveltiunov fyi ☝️ this is related to conversations you have all been involved in previously so would like to get your feedback on this PR if possible
datafusion-expr
crateLogicalPlan
enum to datafusion-expr
crate
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
YOLO! Let's do this
Which issue does this PR close?
Closes #2245
Rationale for this change
We want to move the logical plan to the expr crate so that we can implement subquery expressions (this requires introducing new expressions that reference logical plans).
What changes are included in this PR?
FromStr
implementation onFileType
in the SQL parser and converted intoparse_file_type
method. This is more consistent with other parsing code and fixed a compilation issue.LogicalPlan
. See follow-on issue for more details.Follow-on issues
Are there any user-facing changes?
Yes - code has moved around so this is an API change. I did try to minimize the impact by re-exporting types from their previous location.