From 9487e8689464b224d0f91a0e3c802a8054c8f908 Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Wed, 29 Dec 2021 17:04:50 +0800 Subject: [PATCH 1/8] update doc for IndexMerge GA Signed-off-by: guo-shaoge --- explain-index-merge.md | 4 ++-- optimizer-hints.md | 2 +- system-variables.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/explain-index-merge.md b/explain-index-merge.md index e72aa0487c330..88bc9bbbf27dd 100644 --- a/explain-index-merge.md +++ b/explain-index-merge.md @@ -36,10 +36,10 @@ For the scan operation that is performed on a specific range of data, such as `I > **Note:** > -> At present, the `IndexMerge` feature is disabled by default in TiDB 4.0.0-rc.1. In addition, the currently supported scenarios of `IndexMerge` in TiDB 4.0 are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. Enable the `IndexMerge` in one of two ways: +> At present, the `IndexMerge` feature is disabled by default after v5.4, the currently supported scenarios of `IndexMerge` in TiDB 4.0 are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. Enable the `IndexMerge` in one of two ways: > > - Set `tidb_enable_index_merge=1`; > > - Use the SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) in the query. > -> SQL Hint has a higher priority than system variables. \ No newline at end of file +> SQL Hint has a higher priority than system variables. diff --git a/optimizer-hints.md b/optimizer-hints.md index aa9cb41eb8e0a..24353b7383597 100644 --- a/optimizer-hints.md +++ b/optimizer-hints.md @@ -252,7 +252,7 @@ In addition to this hint, setting the `tidb_enable_index_merge` system variable > **Note:** > -> `NO_INDEX_MERGE` has a higher priority over `USE_INDEX_MERGE`. When both hints are used, `USE_INDEX_MERGE` does not take effect. +> `NO_INDEX_MERGE` has a higher priority over `USE_INDEX_MERGE`. When both hints are used, `USE_INDEX_MERGE` does not take effect. And `NO_INDEX_MERGE` only takes effect in the outermost sub-query block. ### USE_TOJA(boolean_value) diff --git a/system-variables.md b/system-variables.md index 5ff123d7570c1..2d80ddb902d11 100644 --- a/system-variables.md +++ b/system-variables.md @@ -570,7 +570,7 @@ Constraint checking is always performed in place for pessimistic transactions (d ### tidb_enable_index_merge New in v4.0 - Scope: SESSION | GLOBAL -- Default value: `OFF` +- Default value: `ON` - This variable is used to control whether to enable the index merge feature. ### tidb_enable_list_partition New in v5.0 From 972191c19a870d38d648b740b44bb2548b2baba3 Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Wed, 29 Dec 2021 17:09:49 +0800 Subject: [PATCH 2/8] fix Signed-off-by: guo-shaoge --- explain-index-merge.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/explain-index-merge.md b/explain-index-merge.md index 88bc9bbbf27dd..563b5ef3d8bf6 100644 --- a/explain-index-merge.md +++ b/explain-index-merge.md @@ -36,7 +36,7 @@ For the scan operation that is performed on a specific range of data, such as `I > **Note:** > -> At present, the `IndexMerge` feature is disabled by default after v5.4, the currently supported scenarios of `IndexMerge` in TiDB 4.0 are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. Enable the `IndexMerge` in one of two ways: +> At present, the `IndexMerge` feature is enabled by default after v5.4.0, the currently supported scenarios of `IndexMerge` in TiDB 4.0 are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. Enable the `IndexMerge` in one of two ways: > > - Set `tidb_enable_index_merge=1`; > From 88f1ef304f7cc24f36da59fb10fcf580bf5dd310 Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Mon, 3 Jan 2022 18:02:09 +0800 Subject: [PATCH 3/8] fix Signed-off-by: guo-shaoge --- explain-index-merge.md | 13 +++++-------- optimizer-hints.md | 1 - 2 files changed, 5 insertions(+), 9 deletions(-) diff --git a/explain-index-merge.md b/explain-index-merge.md index 563b5ef3d8bf6..9f14e6459307d 100644 --- a/explain-index-merge.md +++ b/explain-index-merge.md @@ -35,11 +35,8 @@ In the above query, the filter condition is a `WHERE` clause that uses `OR` as t For the scan operation that is performed on a specific range of data, such as `IndexRangeScan`/`TableRangeScan`, the `operator info` column in the result has additional information about the scan range compared with other scan operations like `IndexFullScan`/`TableFullScan`. In the above example, the `range:(1,+inf]` in the `IndexRangeScan_13` operator indicates that the operator scans the data from 1 to positive infinity. > **Note:** -> -> At present, the `IndexMerge` feature is enabled by default after v5.4.0, the currently supported scenarios of `IndexMerge` in TiDB 4.0 are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. Enable the `IndexMerge` in one of two ways: -> -> - Set `tidb_enable_index_merge=1`; -> -> - Use the SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) in the query. -> -> SQL Hint has a higher priority than system variables. +> The `IndexMerge` feature is enabled by default after v5.4.0. +> You can use SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) to force the optimizer choose `IndexMerge`. And SQL Hint has a higher priority than system variables. +> The currently supported scenarios of `IndexMerge` in TiDB are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. +> `IndexMerge` is not supported in [tempoaray tables](/temporary-tables.md) for now. +> When there are expressions that cannot be pushed down, SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) is required if you want to use `IndexMerge` diff --git a/optimizer-hints.md b/optimizer-hints.md index af3dc149af3d4..95919ee3d914d 100644 --- a/optimizer-hints.md +++ b/optimizer-hints.md @@ -236,7 +236,6 @@ When multiple `USE_INDEX_MERGE` hints are made to the same table, the optimizer This hint takes effect on strict conditions, including: - If the query can select a single index scan in addition to full table scan, the optimizer does not select index merge. -- If the query is in an explicit transaction, and if the statements before this query has already written data, the optimizer does not select index merge. ## Hints that take effect in the whole query From c64122be7352af4c4617e6bf3e54f81b31084679 Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Thu, 6 Jan 2022 17:04:46 +0800 Subject: [PATCH 4/8] fix comment Signed-off-by: guo-shaoge --- explain-index-merge.md | 2 +- system-variables.md | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/explain-index-merge.md b/explain-index-merge.md index 9f14e6459307d..0c49616c35e90 100644 --- a/explain-index-merge.md +++ b/explain-index-merge.md @@ -35,7 +35,7 @@ In the above query, the filter condition is a `WHERE` clause that uses `OR` as t For the scan operation that is performed on a specific range of data, such as `IndexRangeScan`/`TableRangeScan`, the `operator info` column in the result has additional information about the scan range compared with other scan operations like `IndexFullScan`/`TableFullScan`. In the above example, the `range:(1,+inf]` in the `IndexRangeScan_13` operator indicates that the operator scans the data from 1 to positive infinity. > **Note:** -> The `IndexMerge` feature is enabled by default after v5.4.0. +> The `IndexMerge` feature is enabled by default after v5.4.0 ([`tidb_enable_index_merge`](/system-variables.md#tidb_enable_index_merge-new-in-v40). > You can use SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) to force the optimizer choose `IndexMerge`. And SQL Hint has a higher priority than system variables. > The currently supported scenarios of `IndexMerge` in TiDB are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. > `IndexMerge` is not supported in [tempoaray tables](/temporary-tables.md) for now. diff --git a/system-variables.md b/system-variables.md index a3a811dc96fae..1483c51f1ff8b 100644 --- a/system-variables.md +++ b/system-variables.md @@ -745,6 +745,11 @@ Constraint checking is always performed in place for pessimistic transactions (d - Default value: `ON` - This variable is used to control whether to enable the index merge feature. +> **Note:** +> 1. When upgrade from cluster of version less than v4.0.0 to cluster of version greater equal to v5.4.0, the default value of `tidb_enable_index_merge` is `OFF`. In case of the performance regression because of the changes of execution plan. +> 2. When upgrade from cluster of version greater equal to v4.0.0 to cluster of version greater equal v5.4.0, the default value of `tidb_enable_index_merge` is kept as before upgrade. +> 3. In a new cluster of version greater equal to v5.4.0, the default value of `tidb_enable_index_merge` is `ON`. + ### tidb_enable_list_partition New in v5.0 > **Warning:** From 94706a3dbf964510757e69586b85fcd682ffd3ee Mon Sep 17 00:00:00 2001 From: guo-shaoge Date: Fri, 7 Jan 2022 16:24:13 +0800 Subject: [PATCH 5/8] fix lint Signed-off-by: guo-shaoge --- system-variables.md | 1 + 1 file changed, 1 insertion(+) diff --git a/system-variables.md b/system-variables.md index 1483c51f1ff8b..9fb5b9dccb757 100644 --- a/system-variables.md +++ b/system-variables.md @@ -746,6 +746,7 @@ Constraint checking is always performed in place for pessimistic transactions (d - This variable is used to control whether to enable the index merge feature. > **Note:** +> > 1. When upgrade from cluster of version less than v4.0.0 to cluster of version greater equal to v5.4.0, the default value of `tidb_enable_index_merge` is `OFF`. In case of the performance regression because of the changes of execution plan. > 2. When upgrade from cluster of version greater equal to v4.0.0 to cluster of version greater equal v5.4.0, the default value of `tidb_enable_index_merge` is kept as before upgrade. > 3. In a new cluster of version greater equal to v5.4.0, the default value of `tidb_enable_index_merge` is `ON`. From 4995d779460f1737f3e2fe25010109e52d4ef358 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Fri, 14 Jan 2022 18:59:02 +0800 Subject: [PATCH 6/8] Apply suggestions from code review --- explain-index-merge.md | 12 +++++++----- optimizer-hints.md | 4 +++- system-variables.md | 8 +++++--- 3 files changed, 15 insertions(+), 9 deletions(-) diff --git a/explain-index-merge.md b/explain-index-merge.md index 0c49616c35e90..cbbbac8cf86bf 100644 --- a/explain-index-merge.md +++ b/explain-index-merge.md @@ -35,8 +35,10 @@ In the above query, the filter condition is a `WHERE` clause that uses `OR` as t For the scan operation that is performed on a specific range of data, such as `IndexRangeScan`/`TableRangeScan`, the `operator info` column in the result has additional information about the scan range compared with other scan operations like `IndexFullScan`/`TableFullScan`. In the above example, the `range:(1,+inf]` in the `IndexRangeScan_13` operator indicates that the operator scans the data from 1 to positive infinity. > **Note:** -> The `IndexMerge` feature is enabled by default after v5.4.0 ([`tidb_enable_index_merge`](/system-variables.md#tidb_enable_index_merge-new-in-v40). -> You can use SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) to force the optimizer choose `IndexMerge`. And SQL Hint has a higher priority than system variables. -> The currently supported scenarios of `IndexMerge` in TiDB are limited to the disjunctive normal form (expressions connected by `or`). The conjunctive normal form (expressions connected by `and`) will be supported in later versions. -> `IndexMerge` is not supported in [tempoaray tables](/temporary-tables.md) for now. -> When there are expressions that cannot be pushed down, SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) is required if you want to use `IndexMerge` +> - The Index Merge feature is enabled by default from v5.4.0. That is, [`tidb_enable_index_merge`](/system-variables.md#tidb_enable_index_merge-new-in-v40) is `ON`. +> +> - You can use SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) to force the optimizer to apply Index Merge, regardless of the setting of `tidb_enable_index_merge`. To enable Index Merge when the filtering conditions contain expressions that cannot be pushed down, you must use SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-). +> +> - Index Merge supports only disjunctive normal form (expressions connected by `or`) and does not support conjunctive normal form (expressions connected by `and`). +> +> - Index Merge is not supported in [tempoaray tables](/temporary-tables.md) for now. diff --git a/optimizer-hints.md b/optimizer-hints.md index 95919ee3d914d..b178470851072 100644 --- a/optimizer-hints.md +++ b/optimizer-hints.md @@ -261,7 +261,9 @@ In addition to this hint, setting the `tidb_enable_index_merge` system variable > **Note:** > -> `NO_INDEX_MERGE` has a higher priority over `USE_INDEX_MERGE`. When both hints are used, `USE_INDEX_MERGE` does not take effect. And `NO_INDEX_MERGE` only takes effect in the outermost sub-query block. +> - `NO_INDEX_MERGE` has a higher priority over `USE_INDEX_MERGE`. When both hints are used, `USE_INDEX_MERGE` does not take effect. +> +> - `NO_INDEX_MERGE` only takes effect in the outermost sub-query block. ### USE_TOJA(boolean_value) diff --git a/system-variables.md b/system-variables.md index 9fb5b9dccb757..100ea070f05f7 100644 --- a/system-variables.md +++ b/system-variables.md @@ -747,9 +747,11 @@ Constraint checking is always performed in place for pessimistic transactions (d > **Note:** > -> 1. When upgrade from cluster of version less than v4.0.0 to cluster of version greater equal to v5.4.0, the default value of `tidb_enable_index_merge` is `OFF`. In case of the performance regression because of the changes of execution plan. -> 2. When upgrade from cluster of version greater equal to v4.0.0 to cluster of version greater equal v5.4.0, the default value of `tidb_enable_index_merge` is kept as before upgrade. -> 3. In a new cluster of version greater equal to v5.4.0, the default value of `tidb_enable_index_merge` is `ON`. +> - When upgrading a TiDB cluster from versions earlier than v4.0.0 to v5.4.0 or later, this variable is disabled by default to prevent performance regression due to the changes of an execution plan. +> +> - When upgrading a TiDB cluster from v4.0.0 or later to v5.4.0 or later, this variable remains the setting before the upgrade. +> +> - In TiDB clusters of v5.4.0 and later, this variable is enabled by default. ### tidb_enable_list_partition New in v5.0 From 0b01f6afe2450642ec8fea855fccfce7406134a3 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Thu, 20 Jan 2022 14:24:03 +0800 Subject: [PATCH 7/8] Apply suggestions from code review Co-authored-by: Grace Cai --- explain-index-merge.md | 3 ++- optimizer-hints.md | 3 +-- system-variables.md | 6 +++--- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/explain-index-merge.md b/explain-index-merge.md index cbbbac8cf86bf..7378ce21952f6 100644 --- a/explain-index-merge.md +++ b/explain-index-merge.md @@ -35,9 +35,10 @@ In the above query, the filter condition is a `WHERE` clause that uses `OR` as t For the scan operation that is performed on a specific range of data, such as `IndexRangeScan`/`TableRangeScan`, the `operator info` column in the result has additional information about the scan range compared with other scan operations like `IndexFullScan`/`TableFullScan`. In the above example, the `range:(1,+inf]` in the `IndexRangeScan_13` operator indicates that the operator scans the data from 1 to positive infinity. > **Note:** +> > - The Index Merge feature is enabled by default from v5.4.0. That is, [`tidb_enable_index_merge`](/system-variables.md#tidb_enable_index_merge-new-in-v40) is `ON`. > -> - You can use SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) to force the optimizer to apply Index Merge, regardless of the setting of `tidb_enable_index_merge`. To enable Index Merge when the filtering conditions contain expressions that cannot be pushed down, you must use SQL Hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-). +> - You can use the SQL hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-) to force the optimizer to apply Index Merge, regardless of the setting of `tidb_enable_index_merge`. To enable Index Merge when the filtering conditions contain expressions that cannot be pushed down, you must use the SQL hint [`USE_INDEX_MERGE`](/optimizer-hints.md#use_index_merget1_name-idx1_name--idx2_name-). > > - Index Merge supports only disjunctive normal form (expressions connected by `or`) and does not support conjunctive normal form (expressions connected by `and`). > diff --git a/optimizer-hints.md b/optimizer-hints.md index b178470851072..51292e56457f7 100644 --- a/optimizer-hints.md +++ b/optimizer-hints.md @@ -262,8 +262,7 @@ In addition to this hint, setting the `tidb_enable_index_merge` system variable > **Note:** > > - `NO_INDEX_MERGE` has a higher priority over `USE_INDEX_MERGE`. When both hints are used, `USE_INDEX_MERGE` does not take effect. -> -> - `NO_INDEX_MERGE` only takes effect in the outermost sub-query block. +> - For a subquery, `NO_INDEX_MERGE` only takes effect when it is placed at the outermost level of the subquery. ### USE_TOJA(boolean_value) diff --git a/system-variables.md b/system-variables.md index 100ea070f05f7..f066b353c7187 100644 --- a/system-variables.md +++ b/system-variables.md @@ -747,11 +747,11 @@ Constraint checking is always performed in place for pessimistic transactions (d > **Note:** > -> - When upgrading a TiDB cluster from versions earlier than v4.0.0 to v5.4.0 or later, this variable is disabled by default to prevent performance regression due to the changes of an execution plan. +> - After upgrading a TiDB cluster from versions earlier than v4.0.0 to v5.4.0 or later, this variable is disabled by default to prevent performance regression due to changes of execution plans. > -> - When upgrading a TiDB cluster from v4.0.0 or later to v5.4.0 or later, this variable remains the setting before the upgrade. +> - After upgrading a TiDB cluster from v4.0.0 or later to v5.4.0 or later, this variable remains the setting before the upgrade. > -> - In TiDB clusters of v5.4.0 and later, this variable is enabled by default. +> - Since v5.4.0, for a newly created TiDB cluster, this variable is enabled by default. ### tidb_enable_list_partition New in v5.0 From 31f44042dd4954598835ab492daa53e88fcbcf83 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 24 Jan 2022 11:08:34 +0800 Subject: [PATCH 8/8] Update system-variables.md --- system-variables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index f066b353c7187..b7105bcd3b300 100644 --- a/system-variables.md +++ b/system-variables.md @@ -751,7 +751,7 @@ Constraint checking is always performed in place for pessimistic transactions (d > > - After upgrading a TiDB cluster from v4.0.0 or later to v5.4.0 or later, this variable remains the setting before the upgrade. > -> - Since v5.4.0, for a newly created TiDB cluster, this variable is enabled by default. +> - Since v5.4.0, for a newly deployed TiDB cluster, this variable is enabled by default. ### tidb_enable_list_partition New in v5.0