Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize gc safepoint advancing for internal transaction #32725

Closed
TonsnakeLin opened this issue Mar 2, 2022 · 2 comments · Fixed by #32726 or #32934
Closed

Optimize gc safepoint advancing for internal transaction #32725

TonsnakeLin opened this issue Mar 2, 2022 · 2 comments · Fixed by #32726 or #32934
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@TonsnakeLin
Copy link
Contributor

Enhancement

TiDB advances gc safepoint every tidb_gc_run_interval time with step tidb_gc_life_time. If there is a long time transaction from user client lives more than tidb_gc_life_time, the safpoint can't be advanced until the long transaction is finished or lives than 24h. This mechanism ensures the continuous advancement of gc safepoint and ensures that the data that active transactions need to access will not be cleared.
However, Internal transactions run in TiDB don't comply with the mechanism above. If the internal transaction lives more than tidb_gc_life_time, maybe failed because the data it needs to access was cleared.
Here is an example that an auto analyze task run failed when it lived more than 20 minutes because the data was cleared. The TiDB log printed [tikv:9006]GC life time is shorter than transaction duration, transaction starts at 2021-11-17 14:09:54.756 +0800 CST, GC safe point is 2021-11-17 14:11:54.755 +0800 CST

@jackysp
Copy link
Member

jackysp commented Mar 14, 2022

The most of internal transactions should not exceed 10min. Also, I seem to see that the planner group is trying to expose auto analyze to the public, for example in the processlist at #31724. Does this mean that there is no need to change
the current mechanism? One more question, for analyze it uses max int64 to read the data itself, and it doesn't require high accuracy of data, so is it not worth to let it block GC?

@TonsnakeLin
Copy link
Contributor Author

TonsnakeLin commented Apr 2, 2022

PR #32726 unified all internal transactions startTS and GC advances SafePoint considering them
and I'm sure you add auto analyze to processlist is totally useless to gc advancing safpoint, The ProcessInfo.CurrTxnStartTS always 0.
[2022/04/02 18:03:59.377 +08:00] [INFO] [session.go:3207] [linpinfaxian] [sessionTS=0]
[2022/04/02 18:03:59.468 +08:00] [INFO] [session.go:3207] [linpinfaxian] [sessionTS=0]
@jackysp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
2 participants