Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv: make IntentAgeThreshold a cluster setting #61654

Closed
nvanbenschoten opened this issue Mar 8, 2021 · 0 comments · Fixed by #62519
Closed

kv: make IntentAgeThreshold a cluster setting #61654

nvanbenschoten opened this issue Mar 8, 2021 · 0 comments · Fixed by #62519
Assignees
@aliher1911 @nvanbenschoten
Labels
A-kv-transactions Relating to MVCC and the transactional model. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker

Comments

@nvanbenschoten
Copy link
Member

See #60585 (comment).

Combined with #60619, this should give us a better toolbox to resolve intent buildup issues manually when they come up, until we eliminate the problem entirely.
@nvanbenschoten nvanbenschoten added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv-transactions Relating to MVCC and the transactional model. labels Mar 8, 2021
@cockroachdb cockroachdb deleted a comment from blathers-crl bot Mar 23, 2021
@aliher1911 aliher1911 mentioned this issue Mar 24, 2021
Merged
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Mar 24, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
0b2d06c 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Mar 25, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
3c375dd 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Mar 26, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
81b3274 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Mar 26, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
26bacf7 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
craig bot pushed a commit that referenced this issue Mar 29, 2021
@aliher1911 @maryliag
Merge #62519 #62661
c8e3604 
62519: kvserver: add hidden cluster setting kv.gc.intent_age_threshold r=aliher1911 a=aliher1911

Added hidden cluster setting to change intent age threshold.

Fixes #61654
 
Setting is read on queue level and passed to GC as an arg. This is done to avoid passing cluster setting to GC directly as it currently decoupled from the cluster infrastructure.


62661: ui: show internal statements with option all r=maryliag a=maryliag

Currently on the statements page if the filter selected is `all`, is not showing internal statements
With this fix all statements (internal and from applications) are shown with the `all` filter selected

Release note: Show internal statements on Statements page with filter All selected

Co-authored-by: Oleg Afanasyev <[email protected]>
Co-authored-by: Marylia Gutierrez <[email protected]>
@craig craig bot closed this as completed in 89c91e7 Mar 29, 2021
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Mar 29, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
f0ada23 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
@aliher1911 aliher1911 mentioned this issue Mar 29, 2021
Merged
@aliher1911 aliher1911 mentioned this issue Mar 30, 2021
Merged
@aliher1911 aliher1911 mentioned this issue Mar 30, 2021
Merged
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Mar 31, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
5498af7 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Apr 7, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
780545e 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
aliher1911 added a commit to aliher1911/cockroach that referenced this issue Apr 7, 2021
@aliher1911
kvserver: add hidden cluster setting kv.gc.intent_age_threshold
981d406 
Fixes cockroachdb#61654

Previously intent age threshold for MVCC GC was hardcoded to 2 hours.
It prevents GC resolving intents which are younger that this threshold.

Whenever intents build up, they could affect operations like backup to
proceed efficiently. In that case forcing GC with lower threshold
could help the mitigation.

This change allows changing threshold to a lower value temporarily
and subsequently put a problematic range into GC queue for forced
intent cleanup.

Release note (ops change): New setting to set gc intent age threshold.
@aliher1911 aliher1911 mentioned this issue Apr 23, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aliher1911 aliher1911

@nvanbenschoten nvanbenschoten

Labels
A-kv-transactions Relating to MVCC and the transactional model. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvserver: add hidden cluster setting kv.gc.intent_age_threshold aliher1911/cockroach
2 participants
@aliher1911 @nvanbenschoten