From db849920e5fec77bed74e21fa925cc5b572740dc Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Thu, 7 Dec 2023 21:57:01 -0800 Subject: [PATCH 01/16] Document periodic full compaction. Signed-off-by: Alex Feinberg --- tikv-configuration-file.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 0b88037815056..77b9bd526efff 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -267,6 +267,18 @@ This document only describes the parameters that are not included in command-lin + Sets the size of the connection pool for service and forwarding requests to the server. Setting it to too small a value affects the request latency and load balancing. + Default value: `4` +## raftstore + +#### `periodic-full-compact-start-times` New in 7.6.0" + ++ Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. ++ Default: `[]` (periodic full compaction is disabled by default). + +### `periodic-full-compact-start-max-cpu` New in 7.6.0 + ++ Sets the maximum CPU utilization at which periodic compaction may be run ++ Default value: 0.1 (10%) + ## readpool.unified Configuration items related to the single thread pool serving read requests. This thread pool supersedes the original storage thread pool and coprocessor thread pool since the 4.0 version. From 8406d9e898a7030c02d35d2c8e2accb9522eb29b Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Thu, 7 Dec 2023 22:16:51 -0800 Subject: [PATCH 02/16] Move to existing section. Signed-off-by: Alex Feinberg --- tikv-configuration-file.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 77b9bd526efff..93e958645e4f1 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -267,18 +267,6 @@ This document only describes the parameters that are not included in command-lin + Sets the size of the connection pool for service and forwarding requests to the server. Setting it to too small a value affects the request latency and load balancing. + Default value: `4` -## raftstore - -#### `periodic-full-compact-start-times` New in 7.6.0" - -+ Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. -+ Default: `[]` (periodic full compaction is disabled by default). - -### `periodic-full-compact-start-max-cpu` New in 7.6.0 - -+ Sets the maximum CPU utilization at which periodic compaction may be run -+ Default value: 0.1 (10%) - ## readpool.unified Configuration items related to the single thread pool serving read requests. This thread pool supersedes the original storage thread pool and coprocessor thread pool since the 4.0 version. @@ -1043,6 +1031,16 @@ Configuration items related to Raftstore. + Default value: `0.1` + Minimum value: `0` +#### `periodic-full-compact-start-times` New in 7.6.0" + ++ Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. ++ Default: `[]` (periodic full compaction is disabled by default). + +### `periodic-full-compact-start-max-cpu` New in 7.6.0 + ++ Sets the maximum CPU utilization at which periodic compaction may be run ++ Default value: 0.1 (10%) + ## coprocessor Configuration items related to Coprocessor. From 9519f9fc08a1c926f29f539013fdaef7525c1469 Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Thu, 7 Dec 2023 22:34:26 -0800 Subject: [PATCH 03/16] Fix error. Signed-off-by: Alex Feinberg --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 93e958645e4f1..be87528ba0d78 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1031,7 +1031,7 @@ Configuration items related to Raftstore. + Default value: `0.1` + Minimum value: `0` -#### `periodic-full-compact-start-times` New in 7.6.0" +### `periodic-full-compact-start-times` New in 7.6.0" + Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. + Default: `[]` (periodic full compaction is disabled by default). From 06838ca1ab28b26987c5cac7e2587ba57b0b2852 Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Thu, 7 Dec 2023 22:42:17 -0800 Subject: [PATCH 04/16] Close span. Signed-off-by: Alex Feinberg --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index be87528ba0d78..02cb0de303e87 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1031,7 +1031,7 @@ Configuration items related to Raftstore. + Default value: `0.1` + Minimum value: `0` -### `periodic-full-compact-start-times` New in 7.6.0" +### `periodic-full-compact-start-times` New in 7.6.0 + Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. + Default: `[]` (periodic full compaction is disabled by default). From aabdd8af3b2ea1fb1306f6e0cfa3a4a90aeccebb Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Wed, 13 Dec 2023 18:51:45 -0800 Subject: [PATCH 05/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 02cb0de303e87..b6501e4db5aa2 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1031,7 +1031,7 @@ Configuration items related to Raftstore. + Default value: `0.1` + Minimum value: `0` -### `periodic-full-compact-start-times` New in 7.6.0 +### `periodic-full-compact-start-times` New in v7.6.0 + Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. + Default: `[]` (periodic full compaction is disabled by default). From 8ab307f1eed964aa499574fe66932ed606a4e73a Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Wed, 13 Dec 2023 18:51:51 -0800 Subject: [PATCH 06/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index b6501e4db5aa2..3812fec5aff37 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1036,7 +1036,7 @@ Configuration items related to Raftstore. + Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. + Default: `[]` (periodic full compaction is disabled by default). -### `periodic-full-compact-start-max-cpu` New in 7.6.0 +### `periodic-full-compact-start-max-cpu` New in v7.6.0 + Sets the maximum CPU utilization at which periodic compaction may be run + Default value: 0.1 (10%) From 82b32df38926bdda86ae23d189e39ee707c98a0a Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Wed, 13 Dec 2023 18:51:59 -0800 Subject: [PATCH 07/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 3812fec5aff37..20adb80c2ec95 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1038,7 +1038,7 @@ Configuration items related to Raftstore. ### `periodic-full-compact-start-max-cpu` New in v7.6.0 -+ Sets the maximum CPU utilization at which periodic compaction may be run ++ Controls the CPU usage threshold at which TiKV performs periodic full compaction. + Default value: 0.1 (10%) ## coprocessor From 1b9abf31773a879191e710d44e21756f5f117f5d Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Wed, 13 Dec 2023 18:52:05 -0800 Subject: [PATCH 08/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 20adb80c2ec95..1d75b4b73a962 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1039,7 +1039,7 @@ Configuration items related to Raftstore. ### `periodic-full-compact-start-max-cpu` New in v7.6.0 + Controls the CPU usage threshold at which TiKV performs periodic full compaction. -+ Default value: 0.1 (10%) ++ Default value: 0.1, which means that the maximum CPU usage for periodic compaction processes is 10%. ## coprocessor From 1147993510c698ccddd3e2aef2c9c999d6f42894 Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Wed, 13 Dec 2023 18:52:13 -0800 Subject: [PATCH 09/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 1d75b4b73a962..80f685bc47191 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1033,7 +1033,7 @@ Configuration items related to Raftstore. ### `periodic-full-compact-start-times` New in v7.6.0 -+ Set the times at which periodic full compaction may start. E.g., ``periodic-full-compact-start-times = ["03:00", "23:00"]`` will mean full compaction runs at 03:00 and 23:00 (3am and 11pm) in the tikv node's local timezone. ++ Sets the specific times for initiating periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. + Default: `[]` (periodic full compaction is disabled by default). ### `periodic-full-compact-start-max-cpu` New in v7.6.0 From cdd570e5965f1b262431ae8369e90400e332f319 Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Wed, 13 Dec 2023 18:52:22 -0800 Subject: [PATCH 10/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 80f685bc47191..673ef9627f08a 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1034,7 +1034,7 @@ Configuration items related to Raftstore. ### `periodic-full-compact-start-times` New in v7.6.0 + Sets the specific times for initiating periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. -+ Default: `[]` (periodic full compaction is disabled by default). ++ Default value: `[]`, which means periodic full compaction is disabled by default. ### `periodic-full-compact-start-max-cpu` New in v7.6.0 From 931f4b69bb51cf4e7bb8f3a6c5d8e72d3d71ccb6 Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 14 Dec 2023 16:27:25 +0800 Subject: [PATCH 11/16] fix format --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 673ef9627f08a..49611d9f7e1d3 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1039,7 +1039,7 @@ Configuration items related to Raftstore. ### `periodic-full-compact-start-max-cpu` New in v7.6.0 + Controls the CPU usage threshold at which TiKV performs periodic full compaction. -+ Default value: 0.1, which means that the maximum CPU usage for periodic compaction processes is 10%. ++ Default value: `0.1`, which means that the maximum CPU usage for periodic compaction processes is 10%. ## coprocessor From f2767b0c363392940aba5e428f2e8644c4a19155 Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Fri, 15 Dec 2023 10:06:12 -0800 Subject: [PATCH 12/16] Update dynamic-config.md Signed-off-by: Alex Feinberg --- dynamic-config.md | 1 + 1 file changed, 1 insertion(+) diff --git a/dynamic-config.md b/dynamic-config.md index 424225a568d9d..171f91ebd310b 100644 --- a/dynamic-config.md +++ b/dynamic-config.md @@ -164,6 +164,7 @@ The following TiKV configuration items can be modified dynamically: | `raftstore.apply-max-batch-size` | Raft state machines process data write requests in batches by the BatchSystem. This configuration item specifies the maximum number of Raft state machines that can execute the requests in one batch. | | `raftstore.store-max-batch-size` | Raft state machines process requests for flushing logs into the disk in batches by the BatchSystem. This configuration item specifies the maximum number of Raft state machines that can process the requests in one batch. | | `raftstore.store-io-pool-size` | The number of threads that process Raft I/O tasks, which is also the size of the StoreWriter thread pool (**DO NOT** modify this value from a non-zero value to 0 or from 0 to a non-zero value) | +| `raftstore.periodic-full-compact-start-max-cpu` | The CPU usage threshold at which TiKV performs periodic full compaction if full compaction is enabled | | `readpool.unified.max-thread-count` | The maximum number of threads in the thread pool that uniformly processes read requests, which is the size of the UnifyReadPool thread pool | | `readpool.unified.auto-adjust-pool-size` | Determines whether to automatically adjust the UnifyReadPool thread pool size | | `coprocessor.split-region-on-table` | Enables to split Region by table | From aba3159a885a92d3c31a7c66f18a3ad77e5b9aea Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Mon, 18 Dec 2023 12:46:35 -0800 Subject: [PATCH 13/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 49611d9f7e1d3..211ff38992a56 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1033,6 +1033,10 @@ Configuration items related to Raftstore. ### `periodic-full-compact-start-times` New in v7.6.0 +> **Warning:** +> +> Periodic full compaction is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. + + Sets the specific times for initiating periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. + Default value: `[]`, which means periodic full compaction is disabled by default. From a0d625f8445f96047591a5c152c27fc2c09387b9 Mon Sep 17 00:00:00 2001 From: Alex Feinberg Date: Mon, 18 Dec 2023 12:47:19 -0800 Subject: [PATCH 14/16] Update tikv-configuration-file.md Co-authored-by: Aolin --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 211ff38992a56..f072a86b70fbd 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1037,7 +1037,7 @@ Configuration items related to Raftstore. > > Periodic full compaction is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. -+ Sets the specific times for initiating periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. ++ Sets the specific times for initiating periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. `periodic-full-compact-start-times = ["03:00 +0000", "23:00 +0000"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM in UTC time. + Default value: `[]`, which means periodic full compaction is disabled by default. ### `periodic-full-compact-start-max-cpu` New in v7.6.0 From 7b587359d987ea277fde2a21b25cd829eae5a576 Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 21 Dec 2023 09:51:59 +0800 Subject: [PATCH 15/16] Apply suggestions from code review Co-authored-by: Grace Cai --- tikv-configuration-file.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index f072a86b70fbd..e5d35e7c43215 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1037,12 +1037,12 @@ Configuration items related to Raftstore. > > Periodic full compaction is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. -+ Sets the specific times for initiating periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. `periodic-full-compact-start-times = ["03:00 +0000", "23:00 +0000"]` indicates that TiKV performs full compaction at 03:00 AM and 23:00 PM in UTC time. ++ Sets the specific times that TiKV initiates periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction daily at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. `periodic-full-compact-start-times = ["03:00 +0000", "23:00 +0000"]` indicates that TiKV performs full compaction daily at 03:00 AM and 23:00 PM in UTC time. + Default value: `[]`, which means periodic full compaction is disabled by default. ### `periodic-full-compact-start-max-cpu` New in v7.6.0 -+ Controls the CPU usage threshold at which TiKV performs periodic full compaction. ++ Limits the maximum CPU usage rate for TiKV periodic full compaction. + Default value: `0.1`, which means that the maximum CPU usage for periodic compaction processes is 10%. ## coprocessor From 9575bea1e73bf128a6713b6c658c5f999defdf7b Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 28 Dec 2023 09:50:32 +0800 Subject: [PATCH 16/16] Apply suggestions from code review Co-authored-by: Grace Cai --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index e5d35e7c43215..05e0d41de0d32 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1037,7 +1037,7 @@ Configuration items related to Raftstore. > > Periodic full compaction is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. -+ Sets the specific times that TiKV initiates periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction daily at 03:00 AM and 23:00 PM, based on the local time zone of the TiKV node. `periodic-full-compact-start-times = ["03:00 +0000", "23:00 +0000"]` indicates that TiKV performs full compaction daily at 03:00 AM and 23:00 PM in UTC time. ++ Sets the specific times that TiKV initiates periodic full compaction. You can specify multiple time schedules in an array. For example, `periodic-full-compact-start-times = ["03:00", "23:00"]` indicates that TiKV performs full compaction daily at 03:00 AM and 11:00 PM, based on the local time zone of the TiKV node. `periodic-full-compact-start-times = ["03:00 +0000", "23:00 +0000"]` indicates that TiKV performs full compaction daily at 03:00 AM and 11:00 PM in UTC time. + Default value: `[]`, which means periodic full compaction is disabled by default. ### `periodic-full-compact-start-max-cpu` New in v7.6.0