Reduce the absolute latency of TP statements

[scsldb]

Feature Request

Description

In some mission-critical scenarios, there are dozens of SQL statements in a transaction. And there is a clear requirement for the max latency of these transactions, for example, must under 100ms.

So we should reduce the absolute latency of these CRUD operations.

Category

Performance

Value

Provide high-quality service

Task list

• P0: Support async commit #18220 async commit
  In TiDB we use 2PC to implement the distributed transactions. In 2PC, the first stage is prewrite, and the second stage is commit. We can see the writing statement(INSERT/UPDATE/DELETE) must wait for these two stages finished before return to the client. Async commit means the writing statement can return to the client ASAP the prewrite stage finished, and there are some mechanisms to guarantee the atomic of the distributed transaction. From the client's aspect, the absolute latency of the writing statements is reduced nearly by half.

• P0: Support Clustered Index #4841 cluster index
  Currently, TiDB arranges table's row data by handle id, for example we have a table whose schema is

CRATE TABLE t (
  uid varchar(64),
  s int(10),
  data varchar(255),
  primary key (uid, s),
)

There will be two key-value pairs for each row data:

• handle id -> (uid, s, data)
• (uid, s) -> handle id

The handle id is a uint64 generated by TiDB. There are some disadvantages for this type of data arrangement:

• we need to write two key-value pairs for each row data
• when reading the data by primary key, TiDB will trigger two read operations, one is to fetch the related handle id by primary key, and the other is to fetch the data by the handle id. This will increase the latency of SELECT statements.

In the clustered index, we arrange row data by primary key, in the above example, there will be only one key-value pair for each row data, this will the disadvantages mentioned above:

• (uid, s) -> (data)

Workload estimation

500

Time

GanttStart: 2020-06-15
GanttDue: 2020-08-30
GanttProgress: 95%

[zhangjinpeng87]

/assign