You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-37802][SQL] Composite field name should work with Aggregate push down
### What changes were proposed in this pull request?
Currently, composite filed name such as dept id doesn't work with aggregate push down
sql("SELECT COUNT(\`dept id\`) FROM h2.test.dept")
```
org.apache.spark.sql.catalyst.parser.ParseException:
extraneous input 'id' expecting <EOF>(line 1, pos 5)
== SQL ==
dept id
-----^^^
at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:271)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:132)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parseMultipartIdentifier(ParseDriver.scala:63)
at org.apache.spark.sql.connector.expressions.LogicalExpressions$.parseReference(expressions.scala:39)
at org.apache.spark.sql.connector.expressions.FieldReference$.apply(expressions.scala:365)
at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.translateAggregate(DataSourceStrategy.scala:717)
at org.apache.spark.sql.execution.datasources.v2.PushDownUtils$.$anonfun$pushAggregates$1(PushDownUtils.scala:125)
at scala.collection.immutable.List.flatMap(List.scala:366)
at org.apache.spark.sql.execution.datasources.v2.PushDownUtils$.pushAggregates(PushDownUtils.scala:125)
```
This PR fixes the problem.
### Why are the changes needed?
bug fixing
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
New test
Closes#35108 from huaxingao/composite_name.
Authored-by: Huaxin Gao <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
0 commit comments