Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking Issue For Maintaining Priority Queue In Memory #55063

Closed
Rustin170506 opened this issue Jul 30, 2024 · 0 comments · Fixed by #57385
Closed

Tracking Issue For Maintaining Priority Queue In Memory #55063

Rustin170506 opened this issue Jul 30, 2024 · 0 comments · Fixed by #57385
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement.

Comments

@Rustin170506
Copy link
Member

Enhancement

Currently, we use a priority queue at TiDB to automatically update statistics. However, we currently maintain this queue by rebuilding it every time. This causes the following problems:

  1. When there are many tables, scanning all tables' schema information can have a big impact on the meta information cache. For example, it will load all the tables at once, which is a very slow process and affects other modules' use of the schema cache.
  2. Because we scan it too frequently, we'll spend a lot of CPU on building this queue, since we'll be scanning all the stats cache and the schema cache. See more: High CPU usage for a cluster with large # of tables #49972
    To solve the above issues, we can try to keep the priority queue in memory instead of rebuilding it every time.
@Rustin170506 Rustin170506 added type/enhancement The issue or PR belongs to an enhancement. sig/planner SIG: Planner labels Jul 30, 2024
ti-chi-bot bot pushed a commit that referenced this issue Jul 31, 2024
hawkingrei pushed a commit to hawkingrei/tidb that referenced this issue Aug 1, 2024
@Rustin170506 Rustin170506 added the affects-8.5 This bug affects the 8.5.x(LTS) versions. label Nov 19, 2024
@ti-chi-bot ti-chi-bot bot closed this as completed in cfa52d0 Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant