Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: range scan datum deserialization panicked at 'assertion failed: self.remaining() >= dst.len()' #7158

Closed
Tracked by #6640
lmatz opened this issue Jan 3, 2023 · 13 comments · Fixed by #7191
Closed
Tracked by #6640
Assignees
Labels
type/bug Something isn't working

Comments

@lmatz
Copy link
Contributor

lmatz commented Jan 3, 2023

Describe the bug

thread '<unnamed>' panicked at 'assertion failed: self.remaining() >= dst.len()', /risingwave/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/bytes-1.2.1/src/buf/buf_impl.rs:253:9
--
  | stack backtrace:
  | 0: rust_begin_unwind
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/std/src/panicking.rs:575:5
  | 1: core::panicking::panic_fmt
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/panicking.rs:64:14
  | 2: core::panicking::panic
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/panicking.rs:111:5
  | 3: bytes::buf::buf_impl::Buf::copy_to_slice
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/bytes-1.2.1/src/buf/buf_impl.rs:253:9
  | 4: bytes::buf::buf_impl::Buf::get_i64_le
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/bytes-1.2.1/src/buf/buf_impl.rs:554:9
  | 5: risingwave_common::util::value_encoding::deserialize_value
  | 6: risingwave_common::util::value_encoding::inner_deserialize_datum
  | at ./src/common/src/util/value_encoding/mod.rs:62:19
  | 7: risingwave_common::util::value_encoding::deserialize_datum
  | at ./src/common/src/util/value_encoding/mod.rs:53:5
  | 8: risingwave_batch::executor::row_seq_scan::ScanRange::new::{{closure}}
  | at ./src/batch/src/executor/row_seq_scan.rs:86:21
  | 9: core::iter::adapters::map::map_try_fold::{{closure}}
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/map.rs:91:28
  | 10: core::iter::traits::iterator::Iterator::try_fold
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:2238:21
  | 11: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/map.rs:117:9
  | 12: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/mod.rs:195:9
  | 13: core::iter::traits::iterator::Iterator::try_for_each
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:2299:9
  | 14: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::next
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/mod.rs:178:9
  | 15: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/alloc/src/vec/spec_from_iter_nested.rs:26:32
  | 16: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/alloc/src/vec/spec_from_iter.rs:33:9
  | 17: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/alloc/src/vec/mod.rs:2748:9
  | 18: core::iter::traits::iterator::Iterator::collect
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:1836:9
  | 19: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/result.rs:2075:49
  | 20: core::iter::adapters::try_process
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/mod.rs:164:17
  | 21: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/result.rs:2075:9
  | 22: core::iter::traits::iterator::Iterator::collect
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:1836:9
  | 23: itertools::Itertools::try_collect
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/itertools-0.10.5/src/lib.rs:2014:9
  | 24: risingwave_batch::executor::row_seq_scan::ScanRange::new
  | at ./src/batch/src/executor/row_seq_scan.rs:81:13
  | 25: <risingwave_batch::executor::row_seq_scan::RowSeqScanExecutorBuilder as risingwave_batch::executor::BoxedExecutorBuilder>::new_boxed_executor::{{closure}}::{{closure}}
  | at ./src/batch/src/executor/row_seq_scan.rs:237:39
  | 26: core::iter::adapters::map::map_try_fold::{{closure}}
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/map.rs:91:28
  | 27: core::iter::traits::iterator::Iterator::try_fold
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:2238:21
  | 28: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/map.rs:117:9
  | 29: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/mod.rs:195:9
  | 30: core::iter::traits::iterator::Iterator::try_for_each
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:2299:9
  | 31: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::next
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/mod.rs:178:9
  | 32: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/alloc/src/vec/spec_from_iter_nested.rs:26:32
  | 33: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/alloc/src/vec/spec_from_iter.rs:33:9
  | 34: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/alloc/src/vec/mod.rs:2748:9
  | 35: core::iter::traits::iterator::Iterator::collect
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:1836:9
  | 36: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/result.rs:2075:49
  | 37: core::iter::adapters::try_process
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/adapters/mod.rs:164:17
  | 38: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/result.rs:2075:9
  | 39: core::iter::traits::iterator::Iterator::collect
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/iter/traits/iterator.rs:1836:9
  | 40: itertools::Itertools::try_collect
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/itertools-0.10.5/src/lib.rs:2014:9
  | 41: <risingwave_batch::executor::row_seq_scan::RowSeqScanExecutorBuilder as risingwave_batch::executor::BoxedExecutorBuilder>::new_boxed_executor::{{closure}}
  | at ./src/batch/src/executor/row_seq_scan.rs:235:17
  | 42: <core::pin::Pin<P> as core::future::future::Future>::poll
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/future/future.rs:124:9
  | 43: risingwave_batch::executor::ExecutorBuilder<C>::try_build::{{closure}}
  | at ./src/batch/src/executor/mod.rs:195:29
  | 44: <core::pin::Pin<P> as core::future::future::Future>::poll
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/future/future.rs:124:9
  | 45: risingwave_batch::executor::ExecutorBuilder<C>::build::{{closure}}
  | at ./src/batch/src/executor/mod.rs:179:25
  | 46: risingwave_batch::executor::ExecutorBuilder<C>::try_build::{{closure}}
  | at ./src/batch/src/executor/mod.rs:191:64
  | 47: <core::pin::Pin<P> as core::future::future::Future>::poll
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/future/future.rs:124:9
  | 48: risingwave_batch::executor::ExecutorBuilder<C>::build::{{closure}}
  | at ./src/batch/src/executor/mod.rs:179:25
  | 49: risingwave_batch::task::task_execution::BatchTaskExecution<C>::async_execute::{{closure}}
  | at ./src/batch/src/task/task_execution.rs:273:9
  | 50: risingwave_batch::task::task_manager::BatchManager::fire_task::{{closure}}
  | at ./src/batch/src/task/task_manager.rs:96:37
  | 51: <risingwave_batch::rpc::service::task_service::BatchServiceImpl as risingwave_pb::task_service::task_service_server::TaskService>::create_task::{{closure}}
  | at ./src/batch/src/rpc/service/task_service.rs:73:13
  | 52: <core::pin::Pin<P> as core::future::future::Future>::poll
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/future/future.rs:124:9
  | 53: <risingwave_pb::task_service::task_service_server::TaskServiceServer<T,F> as tower_service::Service<(http::uri::path::PathAndQuery,tonic::request::Request<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<alloc::boxed::Box<dyn core::any::Any+core::marker::Sync+core::marker::Send>,tonic::status::Status>+core::marker::Send>>>)>>::call::{{closure}}
  | at ./src/prost/src/sim/task_service.rs:478:29
  | 54: <core::pin::Pin<P> as core::future::future::Future>::poll
  | at /rustc/bdb07a8ec8e77aa10fb84fae1d4ff71c21180bb4/library/core/src/future/future.rs:124:9
  | 55: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/tracing-0.1.37/src/instrument.rs:272:9
  | 56: madsim_tonic::sim::transport::server::Router<L>::serve_with_shutdown::{{closure}}::{{closure}}::{{closure}}
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/madsim-tonic-0.2.11/src/transport/server.rs:262:56
  | 57: async_task::raw::RawTask<F,T,S>::run
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/async-task-4.3.0/src/raw.rs:511:20
  | 58: madsim::sim::task::Executor::run_all_ready
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/madsim-0.2.12/src/sim/task.rs:209:17
  | 59: madsim::sim::task::Executor::block_on
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/madsim-0.2.12/src/sim/task.rs:159:13
  | 60: madsim::sim::runtime::Runtime::block_on
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/madsim-0.2.12/src/sim/runtime/mod.rs:124:9
  | 61: madsim::sim::runtime::builder::Builder::run::{{closure}}::{{closure}}::{{closure}}
  | at ./.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/madsim-0.2.12/src/sim/runtime/builder.rs:128:35
  | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
  | note: run with `MADSIM_TEST_SEED=34` environment variable to reproduce this error
  | and make sure `MADSIM_CONFIG_HASH=1AD2E4ABC41F3553`

Plan

                                                      QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
 BatchProject { exprs: [1:Int32] }
 └─BatchHashAgg { group_key: [m5.col_0, orders.o_totalprice], aggs: [] }
   └─BatchHashJoin { type: Inner, predicate: m5.col_0::Int64 = orders.o_orderkey }
     ├─BatchExchange { order: [], dist: Single }
     | └─BatchProject { exprs: [m5.col_0, m5.col_0::Int64] }
     |   └─BatchFilter { predicate: ('27538':Varchar::Int16 = m5.col_0) }
     |     └─BatchScan { table: m5, columns: [col_0] }
     └─BatchExchange { order: [], dist: Single }
       └─BatchProject { exprs: [orders.o_totalprice, orders.o_orderkey] }
         └─BatchScan { table: orders, columns: [o_orderkey, o_totalprice], scan_ranges: [o_orderkey = Int32(27538)] }

Local logs from running gist below

compute-node-5688.log
meta-node-5690.log
frontend-4566.log

To Reproduce

CREATE TABLE partsupp (
                          ps_partkey INTEGER,
                          ps_suppkey INTEGER,
                          ps_availqty INTEGER,
                          ps_supplycost NUMERIC,
                          ps_comment VARCHAR,
                          PRIMARY KEY (ps_partkey, ps_suppkey)
);

CREATE TABLE orders (
                        o_orderkey BIGINT,
                        o_custkey INTEGER,
                        o_orderstatus VARCHAR,
                        o_totalprice NUMERIC,
                        o_orderdate DATE,
                        o_orderpriority VARCHAR,
                        o_clerk VARCHAR,
                        o_shippriority INTEGER,
                        o_comment VARCHAR,
                        PRIMARY KEY (o_orderkey)
);

CREATE MATERIALIZED VIEW m5 AS SELECT 0::int AS col_0 FROM partsupp;

SELECT
    1
FROM
    m5 JOIN orders ON m5.col_0 = orders.o_orderkey
GROUP BY
    m5.col_0,
    orders.o_totalprice
HAVING
    SMALLINT '27538' = m5.col_0;

Expected behavior

No response

Additional context

https://buildkite.com/risingwavelabs/main-cron/builds/292#018574f6-e819-41db-ba51-80363d49c625

@lmatz lmatz added the type/bug Something isn't working label Jan 3, 2023
@github-actions github-actions bot added this to the release-0.1.16 milestone Jan 3, 2023
@BugenZhao BugenZhao changed the title bytes: panicked at assertion failed: self.remaining() >= dst.len() bug: range scan datum deserialization panicked Jan 3, 2023
@BugenZhao BugenZhao changed the title bug: range scan datum deserialization panicked bug: range scan datum deserialization panicked at 'assertion failed: self.remaining() >= dst.len()' Jan 3, 2023
@BugenZhao
Copy link
Member

Which query caused this? Is it the last query in the fuzzing.log? 🤔

2022-09-30T18:50:31.045059Z  INFO node{id=11 name="client"}:task{id=830}: risingwave_sqlsmith::runner: Executing: WITH with_0 AS (SELECT t_5.date_time AS col_0, false AS col_1, DATE '2022-09-30' AS col_2, false AS col_3 FROM region AS t_1, m3 AS t_2, region AS t_3, supplier AS t_4, auction AS t_5, customer AS t_8, m2 AS t_9, supplier AS t_10, lineitem AS t_11, m9 AS t_12, partsupp AS t_13, lineitem AS t_14 WHERE true GROUP BY t_10.s_name, t_10.s_acctbal, t_1.r_name, t_5.expires, t_4.s_comment, t_14.l_returnflag, t_10.s_phone, t_8.c_mktsegment, t_14.l_tax, t_4.s_phone, t_11.l_linenumber, t_8.c_acctbal, t_14.l_shipinstruct, t_14.l_extendedprice, t_9.col_0, t_5.date_time, t_11.l_linestatus, t_13.ps_availqty, t_5.item_name, t_14.l_orderkey, t_1.r_regionkey, t_4.s_name, t_14.l_shipmode, t_14.l_quantity, t_14.l_linenumber, t_11.l_suppkey, t_14.l_linestatus, t_3.r_name) SELECT TIMESTAMP '2022-09-30 17:50:31' AS col_0 FROM with_0, hop(auction, auction.date_time, INTERVAL '1', INTERVAL '60') AS hop_15, m4 AS t_16, nation AS t_17, m6 AS t_18, m9 AS t_19, m5 AS t_20 JOIN orders AS t_21 ON t_20.col_0 = t_21.o_orderkey, m1 AS t_22, customer AS t_25, m0 AS t_26, m9 AS t_27 WHERE CASE WHEN true THEN FLOAT '0' WHEN false THEN FLOAT '0' WHEN false THEN FLOAT '51613815.26926697' WHEN true THEN CASE WHEN REAL '1' = FLOAT '634434057.7055044' THEN FLOAT '1691970914.2593095' * ((CASE WHEN true THEN REAL '1' WHEN t_22.col_1 > 206609731.5285713 THEN t_16.col_3 WHEN true THEN REAL '1' WHEN false THEN t_16.col_3 WHEN false THEN t_26.col_1 WHEN false THEN t_16.col_3 ELSE REAL '2147483647' END * t_26.col_1) / (- REAL '1')) WHEN true THEN coalesce(NULL, NULL, NULL, NULL, NULL, NULL, NULL, FLOAT '0', NULL, NULL) WHEN true THEN FLOAT '1341782303.1018193' WHEN true THEN FLOAT '0' WHEN false THEN FLOAT '1' WHEN true THEN FLOAT '949387894.5489348' WHEN false THEN FLOAT '1873357411.294207' ELSE FLOAT '1481248508.5890744' END WHEN false THEN FLOAT '601649305.6033796' WHEN false THEN FLOAT '1' WHEN false THEN FLOAT '2147483647' WHEN false THEN FLOAT '0' ELSE CASE WHEN (TIMESTAMP '2022-09-24 00:51:49' + t_18.col_0) > DATE '2022-09-30' THEN FLOAT '1314125532.5917432' WHEN false THEN FLOAT '1467316773.6668963' WHEN true THEN FLOAT '66616725.859260656' ELSE FLOAT '1' END END < FLOAT '642234493.3729037' GROUP BY t_26.col_1, t_17.n_comment, t_25.c_custkey, t_20.col_0, t_21.o_orderstatus, t_25.c_nationkey, t_17.n_regionkey, hop_15.date_time, t_18.col_1, t_25.c_comment, hop_15.item_name, t_16.col_3, t_21.o_orderpriority, t_18.col_0, t_16.col_0, t_25.c_address, t_21.o_totalprice, t_25.c_mktsegment, t_25.c_name HAVING SMALLINT '27538' = t_20.col_0

@lmatz
Copy link
Contributor Author

lmatz commented Jan 3, 2023

Seem so, didn't find other related context 😟

@kwannoel
Copy link
Contributor

kwannoel commented Jan 3, 2023

Can reproduce locally with this: https://gist.github.com/kwannoel/d70409e6d2e39961cb02a9443deeca9d. Will shrink it further tomorrow. Use shrunk query in issue description.

@kwannoel
Copy link
Contributor

kwannoel commented Jan 3, 2023

Seems related to Meta? (See the full logs in the issue description). Issue with compute node as mentioned by @BugenZhao below.

@BugenZhao
Copy link
Member

Seems related to Meta?

The failure on meta is caused by compute node exiting unexpectedly. The root cause should be panic on the compute node. I guess there's something wrong with the implementation of ser/de of some kind of data type. 🤔

@kwannoel
Copy link
Contributor

kwannoel commented Jan 4, 2023

First bad commit: da6114d from git bisect.

@kwannoel
Copy link
Contributor

kwannoel commented Jan 4, 2023

Changing col_0 to be of type bigint explicitly works:

CREATE TABLE partsupp (
                          ps_partkey INTEGER,
                          ps_suppkey INTEGER,
                          ps_availqty INTEGER,
                          ps_supplycost NUMERIC,
                          ps_comment VARCHAR,
                          PRIMARY KEY (ps_partkey, ps_suppkey)
);

CREATE TABLE orders (
                        o_orderkey BIGINT,
                        o_custkey INTEGER,
                        o_orderstatus VARCHAR,
                        o_totalprice NUMERIC,
                        o_orderdate DATE,
                        o_orderpriority VARCHAR,
                        o_clerk VARCHAR,
                        o_shippriority INTEGER,
                        o_comment VARCHAR,
                        PRIMARY KEY (o_orderkey)
);

CREATE MATERIALIZED VIEW m5 AS SELECT 0::bigint AS col_0 FROM partsupp;

SELECT
    1
FROM
    m5 JOIN orders ON m5.col_0 = orders.o_orderkey
GROUP BY
    m5.col_0,
    orders.o_totalprice
HAVING
    SMALLINT '27538' = m5.col_0;

output

CREATE_TABLE
CREATE_TABLE
CREATE_MATERIALIZED_VIEW
 ?column?
----------
(0 rows)

Range scan is using int64 to deserialize m5.col_0 when it should be using int32:

data: [1, 146, 107, 0, 0]
datatype: Int64
null_tag: 1
thread 'risingwave-main' panicked at 'assertion failed: self.remaining() >= dst.len()', /Users/noelkwan/.cargo/registry/src/github.jparrowsec.cn-1ecc6299db9ec823/bytes-1.2.1/src/buf/buf_impl.rs:253:9

Edit, the query plan is different, may not be the reason, continue investigating...:

                                                          QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------
 BatchProject { exprs: [1:Int32] }
 └─BatchHashAgg { group_key: [m5.col_0, orders.o_totalprice], aggs: [] }
   └─BatchLookupJoin { type: Inner, predicate: m5.col_0 = orders.o_orderkey AND ('27538':Varchar::Int16 = orders.o_orderkey) }
     └─BatchExchange { order: [], dist: Single }
       └─BatchFilter { predicate: ('27538':Varchar::Int16 = m5.col_0) }
         └─BatchScan { table: m5, columns: [col_0] }

@kwannoel
Copy link
Contributor

kwannoel commented Jan 4, 2023

In the query:

CREATE TABLE partsupp (
                          ps_partkey INTEGER,
                          ps_suppkey INTEGER,
                          ps_availqty INTEGER,
                          ps_supplycost NUMERIC,
                          ps_comment VARCHAR,
                          PRIMARY KEY (ps_partkey, ps_suppkey)
);

CREATE TABLE orders (
                        o_orderkey BIGINT,
                        o_custkey INTEGER,
                        o_orderstatus VARCHAR,
                        o_totalprice NUMERIC,
                        o_orderdate DATE,
                        o_orderpriority VARCHAR,
                        o_clerk VARCHAR,
                        o_shippriority INTEGER,
                        o_comment VARCHAR,
                        PRIMARY KEY (o_orderkey)
);

CREATE MATERIALIZED VIEW m5 AS SELECT 0::int AS col_0 FROM partsupp;

SELECT
    1
FROM
    m5 JOIN orders ON m5.col_0 = orders.o_orderkey
GROUP BY
    m5.col_0,
    orders.o_totalprice
HAVING
    SMALLINT '27538' = m5.col_0;

o_order_key is BIGINT.

However in scan_ranges its eq_cond shows a type int32. I guess it should be int64 instead?

                                                      QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
 BatchProject { exprs: [1:Int32] }
 └─BatchHashAgg { group_key: [m5.col_0, orders.o_totalprice], aggs: [] }
   └─BatchHashJoin { type: Inner, predicate: m5.col_0::Int64 = orders.o_orderkey }
     ├─BatchExchange { order: [], dist: Single }
     | └─BatchProject { exprs: [m5.col_0, m5.col_0::Int64] }
     |   └─BatchFilter { predicate: ('27538':Varchar::Int16 = m5.col_0) }
     |     └─BatchScan { table: m5, columns: [col_0] }
     └─BatchExchange { order: [], dist: Single }
       └─BatchProject { exprs: [orders.o_totalprice, orders.o_orderkey] }
         └─BatchScan { table: orders, columns: [o_orderkey, o_totalprice], scan_ranges: [o_orderkey = Int32(27538)] }

Edit: Seems yes, it should be Int64, but conjunctions that LogicalPlan have at some optimization step are wrong. Tracing it.

@BugenZhao
Copy link
Member

cc @chenzl25 @xxchan Could you please take a look? 🥺

@xxchan
Copy link
Member

xxchan commented Jan 4, 2023

We need to add proper cast before predicate move around.

create table t1(x int);
create table t2(x bigint);
insert into t1 values(1);
insert into t2 values(1);
create index i1 on t1(x);
create index i2 on t2(x);

explain select * from i1 join i2 using(x) where x=1;
                                     QUERY PLAN                                      
-------------------------------------------------------------------------------------
 BatchLookupJoin { type: Inner, predicate: i1.x::Int64 = i2.x AND (i2.x = 1:Int32) }
 └─BatchExchange { order: [], dist: Single }
   └─BatchProject { exprs: [i1.x, i1.x::Int64] }
     └─BatchScan { table: i1, columns: [x], scan_ranges: [x = Int32(1)] }
(4 rows)


select * from i1 join i2 using(x) where x=1; -- panic

Hmm actually this triggered a different panic happenning in lookup join

@xxchan
Copy link
Member

xxchan commented Jan 4, 2023

IIRC there are casts when building scan_ranges, so the OP's panic should not occur 🤔

@xxchan
Copy link
Member

xxchan commented Jan 4, 2023

It's the InputRef that has a wrong type 🥸 fixing...

@chenzl25
Copy link
Contributor

chenzl25 commented Jan 4, 2023

It is a bug of derive_predicate_from_eq_condition() which occurs when the equal condition lhs and rhs have two different data types. If @xxchan you don't mind, I can fix it.

@mergify mergify bot closed this as completed in #7191 Jan 4, 2023
mergify bot pushed a commit that referenced this issue Jan 4, 2023
mergify bot pushed a commit that referenced this issue Jan 5, 2023
…7193)

So that bugs like #7158 can be found earlier in frontend instead of backend.

Approved-By: chenzl25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants