Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The threshold of schema apply diff/full is fixed and short #6532

Closed
zanmato1984 opened this issue Dec 23, 2022 · 0 comments · Fixed by #7437
Closed

The threshold of schema apply diff/full is fixed and short #6532

zanmato1984 opened this issue Dec 23, 2022 · 0 comments · Fixed by #7437
Labels
component/storage type/enhancement The issue or PR belongs to an enhancement.

Comments

@zanmato1984
Copy link
Contributor

zanmato1984 commented Dec 23, 2022

Enhancement

The threshold of schema apply diff/full is hard coded as 100. This value could be tight when users are doing highly frequent DDL operations. The problem of this value being small is that, once apply full is triggered, and unluckily there exists tons of tables, the apply diff process could take a long time (for a real world case, 1 to 2 minutes for 60k tables). And given that schema apply process will block read (from getting the schema version) during the whole applying period, this 1 to 2 minutes could result in many read failures.

We may be able to enhance this in some way below:

  1. Make the global schema version an atomic variable to avoid the schema lock, so that schema apply won't block it for long. But we may need to verify the correctness carefully.
  2. Make the threshold relative to current schema version, like 10%, so that for a cluster with too many tables, the apply diff becomes more eager.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/storage type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants