Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression in CASE expression since DF 44 #14277

Closed
Tracked by #14274 ...
andygrove opened this issue Jan 24, 2025 · 4 comments
Closed
Tracked by #14274 ...

Regression in CASE expression since DF 44 #14277

andygrove opened this issue Jan 24, 2025 · 4 comments
Assignees
Labels
bug Something isn't working

Comments

@andygrove
Copy link
Member

Describe the bug

I am trying to upgrade Comet to the latest DataFusion and see some queries fail.

The regression seems to have been introduced in #13953.

Example Query:

SELECT array_remove(array(_2, _3,_4), _2) from t1 where _2 is null

Query plan:

ProjectionExec: expr=[CASE WHEN Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy] IS NULL THEN NULL ELSE array_remove(make_array(Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy], Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_1@1, eval_mode: Legacy], col_2@2), Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy]) END as col_0]
  CometFilterExec: col_0@0 IS NULL
    ScanExec: source=[CometScan parquet  (unknown)], schema=[col_0: Int8, col_1: Int16, col_2: Int32]

Failure:

Invalid argument error: arguments need to have the same data type

Cause:

In the new expr_or_expr function, the following line is causing the failure:

Ok(ColumnarValue::Array(zip(&remainder, &else_, &then_value)?))

The type of else_ is ListArray and the type of then_value is NullArray

Solution:

Perhaps we just need to add a cast in here? I am going to add a repro to DataFusion and work on a fix.

To Reproduce

No response

Expected behavior

No response

Additional context

No response

@andygrove andygrove added the bug Something isn't working label Jan 24, 2025
@andygrove andygrove self-assigned this Jan 24, 2025
@andygrove andygrove changed the title Regression in CASE expression since DF 43 Regression in CASE expression since DF 44 Jan 24, 2025
@andygrove
Copy link
Member Author

@aweltsch I have created a PR with a repro for this issue and wondered if you had thoughts on the best way to fix the regression?

@aweltsch
Copy link
Contributor

aweltsch commented Jan 24, 2025

Hi @andygrove thank you for notifying me of this regression. I took a quick look today and it seems like you are right that casting the then expression might also solve this issue (similar to else expression). I will look into it more in detail tomorrow when I have some more time. Sorry about re-introducing this bug.

@andygrove
Copy link
Member Author

Thanks @aweltsch i appreciate it. I do have a PR up with a fix but it may not be the best fix.

@andygrove
Copy link
Member Author

Using get_coerce_type_for_case_expression resolved the issue in Comet, so I am closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants