Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Page Filtering Incorrectly Handles Pages with Different Row Counts #4744

Closed
tustvold opened this issue Dec 27, 2022 · 1 comment · Fixed by #4743
Closed

Page Filtering Incorrectly Handles Pages with Different Row Counts #4744

tustvold opened this issue Dec 27, 2022 · 1 comment · Fixed by #4743
Assignees
@tustvold
Labels
bug Something isn't working

Comments

@tustvold
Copy link
Contributor

Describe the bug

prune_pages_in_one_row_group uses the size of the previous page when computing the next value of sum_row. This causes it to produce incorrect RowSelection when computing for a row group containing different size pages.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.
@tustvold tustvold added the bug Something isn't working label Dec 27, 2022
@tustvold tustvold self-assigned this Dec 27, 2022
@tustvold tustvold mentioned this issue Dec 27, 2022
Merged
tustvold added a commit to tustvold/arrow-datafusion that referenced this issue Dec 27, 2022
@tustvold
Fix pruning logic (apache#4744)
523466c
@alamb
Copy link
Contributor

alamb commented Dec 27, 2022

Related to #4085

tustvold added a commit that referenced this issue Dec 28, 2022
@tustvold
Simplify parquet filter predicate test, fix Page Filtering Incorrectl…
40bf559 
…y Handles Pages with Different Row Counts (#4743)

* Simplify filter predicate test

* Fix pruning logic (#4744)

* Format

* Fix oom_sort

* Fix tests

* Format

* Clippy

* Clippy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tustvold tustvold

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
2 participants
@alamb @tustvold