Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] [Pyarrow] Pyarrow can not read the file generated by itself when specify use_byte_stream_split #9526

Open
res-life opened this issue Oct 24, 2023 · 1 comment
Open

[BUG] [Pyarrow] Pyarrow can not read the file generated by itself when specify use_byte_stream_split #9526

res-life opened this issue Oct 24, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@res-life
Copy link
Collaborator

Describe the bug
Pyarrow can not read the file generated by itself when specify use_byte_stream_split
It's bug of pyarrow itself.
File this issue to track this pyarrow bug.

Steps/Code to reproduce bug
Run the test case test_parquet_pyarrow_use_byte_stream_split in this PR #9289
Error:

Data size too small for number of values (corrupted file?)

Expected behavior

Environment details (please complete the following information)

Additional context
@res-life res-life added bug Something isn't working ? - Needs Triage Need team to review and classify labels Oct 24, 2023
@res-life res-life mentioned this issue Oct 24, 2023
Merged
@andygrove andygrove self-assigned this Oct 24, 2023
@andygrove
Copy link
Contributor

I'll take a look and see if we can report a bug against Arrow for this

@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Oct 24, 2023
@andygrove andygrove removed their assignment Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@andygrove @mattahrens @res-life