Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tikv: update docs about in-memory pessimistic locks #7938

Merged
merged 1 commit into from
Mar 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion dynamic-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,8 @@ The following TiKV configuration items can be modified online:
| `coprocessor.region-split-keys` | The number of keys in the newly split Region |
| `pessimistic-txn.wait-for-lock-timeout` | The longest duration that a pessimistic transaction waits for the lock |
| `pessimistic-txn.wake-up-delay-duration` | The duration after which a pessimistic transaction is woken up |
| `pessimistic-txn.pipelined` | Whether to enable the pipelined pessimistic locking process |
| `pessimistic-txn.pipelined` | Determines whether to enable the pipelined pessimistic locking process |
| `pessimistic-txn.in-memory` | Determines whether to enable the in-memory pessimistic lock |
| `gc.ratio-threshold` | The threshold at which Region GC is skipped (the number of GC versions/the number of keys) |
| `gc.batch-keys` | The number of keys processed in one batch |
| `gc.max-write-bytes-per-sec` | The maximum bytes that can be written into RocksDB per second |
Expand Down
8 changes: 5 additions & 3 deletions grafana-tikv-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,15 +355,17 @@ This document provides a detailed description of these key metrics on the **TiKV
- Blob GC output file size: The size of Titan GC output file
- Blob GC file count: The count of blob files involved in Titan GC

## Lock manager
## Pessimistic Locking

- Thread CPU: The CPU utilization of the lock manager thread
- Handled tasks: The number of tasks handled by lock manager
- Lock Manager Thread CPU: The CPU utilization of the lock manager thread
- Lock Manager Handled tasks: The number of tasks handled by lock manager
- Waiter lifetime duration: The waiting time of the transaction for the lock to be released
- Wait table: The status information of wait table, including the number of locks and the number of transactions waiting for the lock
- Deadlock detect duration: The time consumed for detecting deadlock
- Detect error: The number of errors encountered when detecting deadlock, including the number of deadlocks
- Deadlock detector leader: The information of the node where the deadlock detector leader is located
- Total pessimistic locks memory size: The memory size occupied by the in-memory pessimistic locks
- In-memory pessimistic locking result: The result of only saving pessimistic locks to memory. `full` means the number of times that the pessimistic lock is not saved to memory because the memory limit is exceeded.

## Memory

Expand Down
35 changes: 30 additions & 5 deletions pessimistic-transaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,28 +63,28 @@ Pessimistic transactions in TiDB behave similarly to those in MySQL. See the min
## Difference with MySQL InnoDB

1. When TiDB executes DML or `SELECT FOR UPDATE` statements that use range in the WHERE clause, concurrent DML statements within the range are not blocked.

For example:

```sql
CREATE TABLE t1 (
id INT NOT NULL PRIMARY KEY,
pad1 VARCHAR(100)
);
INSERT INTO t1 (id) VALUES (1),(5),(10);
```

```sql
BEGIN /*T! PESSIMISTIC */;
SELECT * FROM t1 WHERE id BETWEEN 1 AND 10 FOR UPDATE;
```

```sql
BEGIN /*T! PESSIMISTIC */;
INSERT INTO t1 (id) VALUES (6); -- blocks only in MySQL
UPDATE t1 SET pad1='new value' WHERE id = 5; -- blocks waiting in both MySQL and TiDB
```

This behavior is because TiDB does not currently support _gap locking_.

2. TiDB does not support `SELECT LOCK IN SHARE MODE`.
Expand Down Expand Up @@ -147,3 +147,28 @@ If the TiKV cluster is v4.0.9 or later, you can also dynamically disable this fe
```sql
set config tikv pessimistic-txn.pipelined='false';
```

## In-memory pessimistic lock

In v6.0.0, TiKV introduces the feature of in-memory pessimistic lock. When this feature is enabled, pessimistic locks are usually stored in the memory of the Region leader only, and are not persisted to disk or replicated through Raft to other replicas. This feature can greatly reduce the overhead of acquiring pessimistic locks and improve the throughput of pessimistic transactions.

When the memory usage of in-memory pessimistic locks exceeds the memory threshold of the Region or the TiKV node, the acquisition of pessimistic locks turns to the [pipelined locking process](#pipelined-locking-process). When the Region is merged or the leader is transferred, to avoid the loss of the pessimistic lock, TiKV writes the in-memory pessimistic lock to disk and replicates it to other replicas.

The in-memory pessimistic lock performs similarly to the pipelined locking process, which does not affect the lock acquisition when the cluster is healthy. However, when network isolation occurs in TiKV or a TiKV node is down, the acquired pessimistic lock might be lost.

If the application logic relies on the lock acquiring or lock waiting mechanism, or if you want to guarantee as much as possible the success rate of transaction commits even when the cluster is in an abnormal state, you need to **disable** the in-memory pessimistic lock feature.

This feature is enabled by default. To disable it, modify the TiKV configuration:

```toml
[pessimistic-txn]
in-memory = false
```

To dynamically disable this feature, modify the TiKV configuration online:

{{< copyable "sql" >}}

```sql
set config tikv pessimistic-txn.in-memory='false';
```
8 changes: 7 additions & 1 deletion tikv-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -1536,4 +1536,10 @@ For pessimistic transaction usage, refer to [TiDB Pessimistic Transaction Mode](
### `pipelined`

- This configuration item enables the pipelined process of adding the pessimistic lock. With this feature enabled, after detecting that data can be locked, TiKV immediately notifies TiDB to execute the subsequent requests and write the pessimistic lock asynchronously, which reduces most of the latency and significantly improves the performance of pessimistic transactions. But there is a still low probability that the asynchronous write of the pessimistic lock fails, which might cause the failure of pessimistic transaction commits.
- Default value: `true`
- Default value: `true`

### `in-memory` (New in v6.0.0)

+ Enables the in-memory pessimistic lock feature. With this feature enabled, pessimistic transactions try to store their locks in memory, instead of writing the locks to disk or replicating the locks to other replicas. This improves the performance of pessimistic transactions. However, there is a still low probability that the pessimistic lock gets lost and causes the pessimistic transaction commits to fail.
+ Default value: `true`
+ Note that `in-memory` takes effect only when the value of `pipelined` is `true`.
2 changes: 1 addition & 1 deletion transaction-isolation-levels.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ The Repeatable Read isolation level in TiDB differs from that in MySQL. The MySQ

## Read Committed isolation level

Starting from TiDB [v4.0.0-beta](/releases/release-4.0.0-beta.md#tidb), TiDB supports the Read Committed isolation level.
Starting from TiDB [v4.0.0-beta](/releases/release-4.0.0-beta.md#tidb), TiDB supports the Read Committed isolation level.

For historical reasons, the Read Committed isolation level of current mainstream databases is essentially the [Consistent Read isolation level defined by Oracle](https://docs.oracle.com/cd/B19306_01/server.102/b14220/consist.htm). In order to adapt to this situation, the Read Committed isolation level in TiDB pessimistic transactions is also a consistent read behavior in essence.

Expand Down