refactor(parquet): Use velox parquet reader in Metadatatest #11472
+134
−20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
3 times, most recently
from
4af254d to
c97061c
Compare
majetideepak
reviewed
Nov 8, 2024
There was a problem hiding this comment.
@jkhaliqi can you move the file from arrow tests to here so that we can only see the Velox specific changes?
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
4 times, most recently
from
d62718d to
495598e
Compare
majetideepak
reviewed
Nov 9, 2024
majetideepak
reviewed
Nov 9, 2024
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
6 times, most recently
from
8aa9724 to
85013aa
Compare
jkhaliqi
changed the title
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
10 times, most recently
from
287d618 to
d5f7423
Compare
jkhaliqi
changed the title
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
from
d5f7423 to
ece201e
Compare
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
2 times, most recently
from
abc8f1d to
c862477
Compare
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
7 times, most recently
from
4642cc1 to
38f48d1
Compare
velox/dwio/parquet/reader/Metadata.h
Outdated
| @@ -18,6 +18,7 @@ | |||
|
|
|||
| #include "velox/dwio/common/Statistics.h" | |||
| #include "velox/dwio/common/compression/Compression.h" | |||
| #include "velox/dwio/parquet/thrift/ParquetThriftTypes.h" | |||
| #include "velox/dwio/parquet/reader/ParquetReader.h" | ||
| #include "velox/exec/tests/utils/TempFilePath.h" | ||
|
|
||
| #include <arrow/io/api.h> |
There was a problem hiding this comment.
We should not need this #include "arrow/util/key_value_metadata.h" above as well.
There was a problem hiding this comment.
Looks like the above #include "arrow/util/key_value_metadata.h" is used for a test with the arrow writer. Since this PR is focusing on refactoring the arrow reader to parquet reader (issue). Should it be fine to leave it for now, and change it when a refactor for the arrow writer to velox writer happens?
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
2 times, most recently
from
f725aa9 to
dde2063
Compare
jkhaliqi
changed the title
jkhaliqi
force-pushed
the
rmv_arrow_parquet_reader
branch
from
dde2063 to
5debd1a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.