Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Add the metrics for WriteFilesExecTransformer #4214

Merged
merged 6 commits into from
Jan 9, 2024

Conversation

JkSelf
Copy link
Contributor

@JkSelf JkSelf commented Dec 28, 2023

What changes were proposed in this pull request?

Add the physicalWrittenBytes and numWrittenFiles metrics for WriteFilesExecTransformer.

image

How was this patch tested?

Adding Unit test and UI locally check.

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

Run Gluten Clickhouse CI

3 similar comments
Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

@JkSelf JkSelf requested review from ulysses-you and rui-mo January 2, 2024 01:22
@zhouyuan zhouyuan changed the title Add the metrics for WriteFilesExecTransformer [VL] Add the metrics for WriteFilesExecTransformer Jan 2, 2024
Copy link

github-actions bot commented Jan 2, 2024

Run Gluten Clickhouse CI

2 similar comments
Copy link

github-actions bot commented Jan 2, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Jan 2, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Jan 3, 2024

Run Gluten Clickhouse CI

Copy link
Contributor

@rui-mo rui-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noticed some conflicts. Could you resolve?

Copy link

github-actions bot commented Jan 3, 2024

Run Gluten Clickhouse CI

1 similar comment
Copy link

github-actions bot commented Jan 3, 2024

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Jan 9, 2024

Run Gluten Clickhouse CI

@JkSelf
Copy link
Contributor Author

JkSelf commented Jan 9, 2024

@ulysses-you @rui-mo Do you have any further comment? Thanks.

@JkSelf JkSelf merged commit 75d079c into apache:main Jan 9, 2024
19 checks passed
Copy link
Contributor

@rui-mo rui-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4214_time.csv log/native_master_01_08_2024_b81ecdef1_time.csv difference percentage
q1 33.32 33.20 -0.120 99.64%
q2 25.42 25.74 0.316 101.24%
q3 39.21 38.04 -1.166 97.03%
q4 39.71 39.01 -0.695 98.25%
q5 72.11 71.87 -0.237 99.67%
q6 7.08 8.18 1.094 115.45%
q7 83.52 85.51 1.982 102.37%
q8 86.53 85.98 -0.554 99.36%
q9 122.69 122.63 -0.065 99.95%
q10 42.56 43.71 1.151 102.70%
q11 20.28 20.14 -0.147 99.28%
q12 26.42 27.54 1.115 104.22%
q13 46.40 46.35 -0.052 99.89%
q14 19.09 18.04 -1.047 94.51%
q15 29.55 28.07 -1.489 94.96%
q16 15.50 15.33 -0.167 98.92%
q17 158.08 157.40 -0.671 99.58%
q18 195.33 196.16 0.832 100.43%
q19 16.45 16.55 0.104 100.63%
q20 27.48 29.10 1.623 105.91%
q21 227.52 226.46 -1.057 99.54%
q22 13.79 14.01 0.223 101.62%
total 1348.04 1349.01 0.971 100.07%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants