Add Redis table Engine/Function

[JackyWoo]

This PR try to close #46146

Changelog category (leave one):

• New Feature

Implements storage in the Redis.
Usage : ENGINE = Redis(host:port[, db_index[, password[, pool_size]]]) PRIMARY KEY(key)

In current design, redis table engine/function must have only one column as primary key who is redis key. And the other columns will be serialized together as redis value.

Redis table engine/function now supports

1. select, insert, update, delete, truncate operations.
2. equals, in predictes pushdown.

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

1. Add TableFunctionRedis
2. Add table engine Redis
3. Add RedisCommon which contains Redis related tools and types
4. Support equals and in filter push down into Redis

Documentation entry for user-facing changes

• Documentation is written (mandatory for new features)

Tests

• Integration tests

[robot-ch-test-poll3]

This is an automated comment for commit 49082bf with description of existing statuses. It's updated for the latest CI running
The full report is available here
The overall status of the commit is 🟢 success

Check name	Description	Status
CI running	A meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR	🟢 success
Docs Check	Builds and tests the documentation	🟢 success
Fast test	Normally this is the first check that is ran for a PR. It builds ClickHouse and runs most of stateless functional tests, omitting some. If it fails, further checks are not started until it is fixed. Look at the report to see which tests fail, then reproduce the failure locally as described here	🟢 success
Mergeable Check	Checks if all other necessary checks are successful	🟢 success
Push to Dockerhub	The check for building and pushing the CI related docker images to docker hub	🟢 success
Style Check	Runs a set of checks to keep the code style clean. If some of tests failed, see the related log from the report	🟢 success

[helifu]

Sorry, I'm interested in this feature, but could you please share the usage scenarious of it? Thanks in advance :)

[JackyWoo]

@helifu n my production scenario, there is a need for flexibility in querying as well as a requirement for high query throughput. I want to promote query throughput Redis.

Usage:

1. Insert-into-select data to Redis and query thought Redis directly.
2. Insert data to Redis and query throught ClickHouse.

[helifu]

Ah, that's cool. But it seems that redis can't offer high throughput since it's a key-value storage IIRC.

[JackyWoo]

It will work in key based query scenarios.

[JackyWoo]

@pufit Please review it when you have a chance. There are some test errors but maybe unrelated with the PR.

[JackyWoo]

@pufit May I ask when you have a chance to review th PR?

[pufit]

Besides some small questions, LGTM!

[JackyWoo]

@pufit for the issue duplicated keys.

To perfectly resolve it, we need additional rehashing stat info from Redis, I submit an issue to Redis for it. And now we have two ways:

1. Just leave it and add notifications in docs. For duplication is a vary rare case and we can fix it when Redis can provide rehasing stat info.
2. Keep two scan results in memory and filter them every scan.

I prefer the second one, but any way I will take your opinion?

[pufit]

@pufit for the issue duplicated keys.

To perfectly resolve it, we need additional rehashing stat info from Redis, I submit an issue to Redis for it. And now we have two ways:

1. Just leave it and add notifications in docs. For duplication is a vary rare case and we can fix it when Redis can provide rehasing stat info.
2. Keep two scan results in memory and filter them every scan.

I prefer the second one, but any way I will take your opinion?

Well, actually, the first option looks much more straightforward. All duplications can be handled on ClickHouse side (using DISTINCT, for example). And we can improve this behavior in the future if we consider it necessary.

[JackyWoo]

@pufit Thank you, docs is modified.

[pufit]

Failed tests:
test_async_query_sending - #50480
02479_mysql_connect_to_self - #50911

[gingerwizard]

@JackyWoo late to see this but started testing it. I see you always use MGET command and never HMGET - the latter feels more natural for table data. Anyway we could support this as MGET means the values must be a string?

[JackyWoo]

@gingerwizard Assume that there is a table with schama (key, v1, v2, v3) in which key is ClickHouse primary key. I map key to Redis key and the other columns are packed to Redis value. So MGET is enough. You can start with docs.

PS: key, v1, v2, v3 can be any ClickHouse data type.