forward compability 23.8 -> 23.4 (CANNOT_READ_ALL_DATA / TOO_LARGE_STRING_SIZE) / ratio_of_defaults_for_sparse_serialization

[den-crane]

it's not possible to downgrade to any version before 23.5

apt-get install clickhouse-server=23.8.3.48 clickhouse-client=23.8.3.48 clickhouse-common-static=23.8.3.48
/etc/init.d/clickhouse-server restart

create table test(event_date Date, event_time DateTime, exception String) 
ENGINE = MergeTree
PARTITION BY toYYYYMM(event_date)
ORDER BY (event_date, event_time);

insert into test(event_date) select today() from numbers(10);

select * from test;
┌─event_date─┬──────────event_time─┬─exception─┐
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
│ 2023-09-29 │ 1970-01-01 00:00:00 │           │
└────────────┴─────────────────────┴───────────┘


apt-get install clickhouse-server=23.4.6.25 clickhouse-client=23.4.6.25 clickhouse-common-static=23.4.6.25
/etc/init.d/clickhouse-server restart

select * from test;

Received exception from server (version 23.4.6):
Code: 33. DB::Exception: Received from localhost:9000. 
DB::Exception: Cannot read all data in MergeTreeReaderCompact. 
Rows read: 4. Rows expected: 10.: (while reading column event_time): 
While executing MergeTreeInOrder. (CANNOT_READ_ALL_DATA)

select exception from test;

Received exception from server (version 23.4.6):
Code: 131. DB::Exception: Received from localhost:9000. 
DB::Exception: Too large string size: 4611686018427387914. 
The maximum is: 17179869184.: (while reading column exception): 
While executing MergeTreeInOrder. (TOO_LARGE_STRING_SIZE)

---

no issue if ratio_of_defaults_for_sparse_serialization = 1

23.8.3.48

CREATE TABLE test
(
    `event_date` Date,
    `event_time` DateTime,
    `exception` String
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(event_date)
ORDER BY (event_date, event_time)
SETTINGS ratio_of_defaults_for_sparse_serialization = 1

But it does not fix (not a workaround) for 23.4.

[den-crane]

cc @CurtizJ

[den-crane]

Actually the solution for a downgrade is to set after the downgrade

cat /etc/clickhouse-server/config.d/config_substitutes.xml
<?xml version="1.0"?>
<clickhouse>
    <merge_tree>
     <ratio_of_defaults_for_sparse_serialization>0.9375</ratio_of_defaults_for_sparse_serialization>
    </merge_tree>
</clickhouse>

#49660