Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DataFusion serialization doesn't handle ScalarValue::Dictionary, Binary, LargeBinary, Time64, IntervalMonthDayNano, Struct #3531

Closed
6 tasks done
alamb opened this issue Sep 19, 2022 · 1 comment · Fixed by #3536
Closed
6 tasks done

DataFusion serialization doesn't handle ScalarValue::Dictionary, Binary, LargeBinary, Time64, IntervalMonthDayNano, Struct #3531

alamb opened this issue Sep 19, 2022 · 1 comment · Fixed by #3536
Assignees
@alamb
Labels
bug Something isn't working

Comments

@alamb
Copy link
Contributor

alamb commented Sep 19, 2022 •
edited
Loading

Describe the bug

While updating to the latest and greatest DataFusion pin in IOx (https://github.com/influxdata/influxdb_iox/pull/5672) I got the following error:

Error encoding expr as protobuf: Dictionary(Int32, Utf8(\"B\")) is invalid as a DataFusion scalar value"

I think the issue is that now more data type casting is done as part of logical planning (#3396) the types of some constants (ScalarValues ) have been coerced to the correct type resulting in serialized ScalarValue::Dictionary that were not there before

To Reproduce
Try to serialize/deserialize ScalarValue::Dictionary

Expected behavior
it works without error!

Additional context
https://github.com/influxdata/influxdb_iox/pull/5672

Types that need support:
@alamb alamb added the bug Something isn't working label Sep 19, 2022
@alamb alamb self-assigned this Sep 19, 2022
@alamb
Copy link
Contributor Author

alamb commented Sep 19, 2022

I will fix this

@alamb alamb changed the title DataFusion serialization doesn't handle ScalarValue::Dictionary types DataFusion serialization doesn't handle ScalarValue::Dictionary or ScalarValue::Binary ScalarValue::LargeBinary ScalarValue::Time64 ScalarValue::IntervalMonthDayNano ScalarValue::Struct Sep 19, 2022
@alamb alamb changed the title DataFusion serialization doesn't handle ScalarValue::Dictionary or ScalarValue::Binary ScalarValue::LargeBinary ScalarValue::Time64 ScalarValue::IntervalMonthDayNano ScalarValue::Struct DataFusion serialization doesn't handle ScalarValue::Dictionary, Binary, LargeBinary, Time64, IntervalMonthDayNano, Struct Sep 19, 2022
This was referenced Sep 19, 2022
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alamb alamb

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add serialization of ScalarValue::Struct alamb/datafusion
1 participant
@alamb