You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Comet encountered some errors on Spark test like following. It is because DataFusion SortExec calls lexsort_to_indices and the function fallbacks to sort_to_indices for single column case. However, sort_to_indices doesn't support all data types, e.g., struct. For unsupported data types, lexsort_to_indices should not fallback to sort_to_indices.
[info] - vectorized reader: struct *** FAILED *** (515 milliseconds)
[info] org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 644.0 failed 1 times, most recent failure: Lost task 0.0 in stage 644.0 (TID 862) (192.168.86.44 executor driver): org.apache.comet.CometNa
tiveException: Compute error: Sort not supported for data type Struct([Field { name: "_1", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "_2", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }])
To Reproduce
Expected behavior
Additional context
The text was updated successfully, but these errors were encountered:
Describe the bug
Comet encountered some errors on Spark test like following. It is because DataFusion SortExec calls
lexsort_to_indices
and the function fallbacks tosort_to_indices
for single column case. However,sort_to_indices
doesn't support all data types, e.g., struct. For unsupported data types,lexsort_to_indices
should not fallback tosort_to_indices
.To Reproduce
Expected behavior
Additional context
The text was updated successfully, but these errors were encountered: