Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update TiDB 6MB limit #5676

Merged
merged 9 commits into from
Jun 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion best-practices/tidb-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ TiDB provides complete distributed transactions and the model has some optimizat
As distributed transactions need to conduct two-phase commit and the bottom layer performs Raft replication, if a transaction is very large, the commit process would be quite slow, and the following Raft replication process is thus stuck. To avoid this problem, the transaction size is limited:

- A transaction is limited to 5,000 SQL statements (by default)
- Each Key-Value entry is no more than 6 MB
- Each Key-Value entry is no more than 6 MB (by default)
- The total size of Key-Value entries is no more than 10 GB.

You can find similar limits in [Google Cloud Spanner](https://cloud.google.com/spanner/quotas).
Expand Down
4 changes: 2 additions & 2 deletions error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,9 +88,9 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the

* Error Number: 8025

The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB is 6 MB.
The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB is 6 MB by default.

If a pair exceeds this limit, you need to manually deal with this row of data to meet the 6 MB limit.
If a pair exceeds this limit, you need to properly adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration value to relax the limit.

* Error Number: 8026

Expand Down
2 changes: 1 addition & 1 deletion faq/migration-tidb-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ The total read capacity has no limit. You can increase the read capacity by addi

### The error message `transaction too large` is displayed

Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB.
Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration value up to 120MB.

Distributed transactions need two-phase commit and the bottom layer performs the Raft replication. If a transaction is very large, the commit process would be quite slow and the write conflict is more likely to occur. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, modify the value of `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to 10G. The actual limitation is also affected by the physical memory of the machine.

Expand Down
2 changes: 1 addition & 1 deletion storage-engine/titan-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Titan is suitable for the scenarios where a huge volume of data is written to th

The prerequisites for enabling Titan are as follows:

- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 6 MB due to the limitation of the TiKV Raft layer.
- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) configuration value to relax the limit.
- No range query will be performed or you do not need a high performance of range query. Because the data stored in Titan is not well-ordered, its performance of range query is poorer than that of RocksDB, especially for the query of a large range. According PingCAP's internal test, Titan's range query performance is 40% to a few times lower than that of RocksDB.
- Sufficient disk space, because Titan reduces write amplification at the cost of disk space. In addition, Titan compresses values one by one, and its compression rate is lower than that of RocksDB. RocksDB compresses blocks one by one. Therefore, Titan consumes more storage space than RocksDB, which is expected and normal. In some situations, Titan's storage consumption can be twice that of RocksDB.

Expand Down
2 changes: 1 addition & 1 deletion tidb-limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ This document describes the common usage limitations of TiDB, including the maxi

| Type | Upper limit |
|:----------|:----------|
| Size | 6 MB |
| Size | 6 MB by default. You can adjust the size limit via the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item. |

## Limitation on a single column

Expand Down