You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently, the protobuf serialisation does not support Arrow Map types. This means that any query from ballista on, say, a Parquet file that contains Map data structures will fail to serialise the Arrow Schema, and hence lead to failure of the whole query (regardless of whether the query actually touches the Maps in the Parquet).
Describe the solution you'd like
Add the missing functionality to serialise Arrow Map data types to datafusion/proto/proto/datafusion.proto and the related serialisers.
Describe alternatives you've considered
I could not see any alternatives.
Additional context
Using this parquet file: map.parquet.gz
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Internal("failed to serialize logical plan: Plan(\"General error: Proto serialization error: The Map data type is not yet supported\")")',
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently, the protobuf serialisation does not support Arrow Map types. This means that any query from
ballista
on, say, a Parquet file that contains Map data structures will fail to serialise the Arrow Schema, and hence lead to failure of the whole query (regardless of whether the query actually touches the Maps in the Parquet).Describe the solution you'd like
Add the missing functionality to serialise Arrow Map data types to
datafusion/proto/proto/datafusion.proto
and the related serialisers.Describe alternatives you've considered
I could not see any alternatives.
Additional context
Using this parquet file:
map.parquet.gz
and the following code:
will fail with:
A correct execution should produce:
The text was updated successfully, but these errors were encountered: