Tracking issue for Dynamic Configuration Change

[rleungx]

The PR #13660 proposes a unified way to manage the configuration options of TiDB, TiKV, and PD by storing them in PD and support dynamically change the configuration options by using the same way among the different components which can greatly improve usability.

Note

This issue is a TODO list catalog that is used to clarify the split details of the entire feature. If you are interested in a part, use the following workflow:

1. Select the module you are interested in.
2. Create a new issue to claim the relevant work and describe the rough implementation in the new issue.
3. File a new pull request.

Issues:

• Protocol Definition

  • Define the Configuration gRPC Service and the related message type in kvproto #498 @overvenus

• Basic Framework

  • PD @rleungx
    • Implement Create function #1977 @rleungx
    • Implement Get function #1977 @rleungx
    • Implement Update function #1977 @rleungx
    • Implement Delete function #1977 @rleungx
    • Support persist and reload configurations #1977 @rleungx
    • Finish glue code for PD component (to make PD configuration management consistent with other components)
  • TiKV @NingLin-P
    • Implement protocol of communicate with PD @NingLin-P
    • Implement config update workflow @NingLin-P
    • Support change config of raft store module @NingLin-P
    • Support change config of split checker module @NingLin-P
    • Support change config of coprocessor module
    • Support change config of rocksdb module
    • Support change config of gc worker module @NingLin-P
    • Support change config of lock manager module @NingLin-P
  • TiDB
    • Add system table to manage the changed configuration
    • Replace current way (HTTP) with gRPC to get all cluster configuration (need to finish client first)
    • Implement the new start process

• Client implementation

  • Go (take PD client as an example) #1977 @rleungx
  • Rust

• Improvement

  • PD
    • Support more transition of type for toml (details can be found here)
    • Handle the error more properly #1977 @rleungx
    • Define error type #1977 @rleungx

• Optional

  • Add HTTP API for PD to manage configuration

• Testing

  • Upgrade tests
  • Client implementation tests

Teachability, Documentation, Adoption, Migration Strategy:

Proposal: #13660