Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add get_window_frame in window_expr, show frame info in window_agg_exec #4508

Merged
merged 2 commits into from
Dec 6, 2022

Conversation

Ted-Jiang
Copy link
Member

@Ted-Jiang Ted-Jiang commented Dec 5, 2022

Signed-off-by: yangjiang [email protected]

Which issue does this PR close?

Closes #4509 .

Rationale for this change

Need show window frame info in window_agg_exec
Before

explain SELECT s_suppkey, s_nationkey, s_acctbal, (sum(s_acctbal) over (
                                                                   ORDER BY s_acctbal DESC ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)) wr
   FROM supplier

limit 10;
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                                                                                                                                                             |
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan  | Projection: supplier.s_suppkey, supplier.s_nationkey, supplier.s_acctbal, SUM(supplier.s_acctbal) ORDER BY [supplier.s_acctbal DESC NULLS FIRST] ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING AS wr                                  |
|               |   Limit: skip=0, fetch=10                                                                                                                                                                                                        |
|               |     WindowAggr: windowExpr=[[SUM(supplier.s_acctbal) ORDER BY [supplier.s_acctbal DESC NULLS FIRST] ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING]]                                                                                   |
|               |       TableScan: supplier projection=[s_suppkey, s_nationkey, s_acctbal]                                                                                                                                                         |
| physical_plan | ProjectionExec: expr=[s_suppkey@1 as s_suppkey, s_nationkey@2 as s_nationkey, s_acctbal@3 as s_acctbal, SUM(supplier.s_acctbal) ORDER BY [supplier.s_acctbal DESC NULLS FIRST] ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING@0 as wr] |
|               |   GlobalLimitExec: skip=0, fetch=10                                                                                                                                                                                              |
|               |     WindowAggExec: wdw=[SUM(supplier.s_acctbal): Ok(Field { name: "SUM(supplier.s_acctbal)", data_type: Float64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: None })]                                          |
|               |       SortExec: [s_acctbal@2 DESC]                                                                                                                                                                                               |
|               |         ParquetExec: limit=None, partitions=[**], projection=[s_suppkey, s_nationkey, s_acctbal]             |
|               |                                                                                                                                                                                                                                  |
+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set. Query took 0.192 seconds.

Now

+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                                                                                                                                                                                                                                |
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 ***                                                                                                                                                                                                                                                            |
|               |     WindowAggExec: wdw=[SUM(supplier.s_acctbal): Ok(Field { name: "SUM(supplier.s_acctbal)", data_type: Float64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), frame: Some(WindowFrame { units: Rows, start_bound: Preceding(UInt64(1)), end_bound: Following(UInt64(1)) })] |
|               |       SortExec: [s_acctbal@2 DESC]                                                                                                                                                                                                                                                                  ***
+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set. Query took 0.019 seconds.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added core Core DataFusion crate physical-expr Physical Expressions labels Dec 5, 2022
@Ted-Jiang Ted-Jiang changed the title add get_window_frame in window_expr, show frame info in window_agg_exec Add get_window_frame in window_expr, show frame info in window_agg_exec Dec 5, 2022
@Ted-Jiang Ted-Jiang requested a review from alamb December 5, 2022 05:14
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -- thanks @Ted-Jiang

I wonder if #4516 is sufficient?

Signed-off-by: yangjiang <[email protected]>

# Conflicts:
#	datafusion/core/tests/sql/window.rs
@Ted-Jiang
Copy link
Member Author

LGTM -- thanks @Ted-Jiang

I wonder if #4516 is sufficient?

@alamb Thanks for remind me, this change is for showing more info in physical node, still need this after #4516 .

Signed-off-by: yangjiang <[email protected]>
@alamb alamb merged commit 0d63c68 into apache:master Dec 6, 2022
@ursabot
Copy link

ursabot commented Dec 6, 2022

Benchmark runs are scheduled for baseline = bfd41a3 and contender = 0d63c68. 0d63c68 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Show window frame info in physical plan
3 participants