Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
109833: kvflowcontrol: annotate/fix perf regressions r=irfansharif a=irfansharif

- Replace the flow controller level mutex-backed kvflowcontrol.Stream => token bucket map with sync.Map. On kv0/enc=false/nodes=3/cpu=96 accessing this map contributed to a high amount of mutex contention. We observe that this bucket is effectively read-only - entries for keys are written once (on creation) and read frequently after. We don't currently GC these buckets, but even if we did, the same access pattern would hold. We'll note that using a sync.Map is slightly more expensive CPU-wise.
- Replace various map accesses with individual variables. We were needly using maps to access one of two variables, keyed by work class, for example when maintaining metrics per work class, or tracking token adjustments. The map accesses appeared prominently in CPU profiles and was unnecessary overhead.
- Avoid using log.ExpensiveLogEnabled in hot code paths; it shows up in CPU profiles.
- Slightly reduce the surface area of kvflowhandle.Handle.mu when returning flow tokens.
- We also annotate various other points in the code where peep-hole optimizations exist, as surfaced by kv0/enc=false/nodes=3/cpu=96.

Part of #104154.

Release note: None

110088: privilege: automate generation of ByName map r=ecwall a=andyyang890

This patch automates the process of generating the `ByName` map so that
any newly added privileges will automatically be included.

Epic: None

Release note: None

110110: admission: add metric for bypassed IO admission work r=irfansharif a=irfansharif

Part of #82743. We introduce an admission.granter.io_tokens_bypassed.kv metric, that tracks the total number of tokens taken by work bypassing admission control. For example, follower writes without flow control.

Aside: #109640 ripped out a tokens-taken-without-permission metric that was supposed to capture some of this, but even for standard admission work we'd routinely exercise that code path. When admitting work, we take 1 token, and later take the remaining without permission.

Release note: None

Co-authored-by: irfan sharif <[email protected]>
Co-authored-by: Andy Yang <[email protected]>
  • Loading branch information
3 people committed Sep 6, 2023
4 parents c0f0059 + e571ffe + 99fde7d + 6dba9cc commit 2b6e80e
Show file tree
Hide file tree
Showing 18 changed files with 319 additions and 197 deletions.
3 changes: 2 additions & 1 deletion pkg/kv/kvserver/kvflowcontrol/kvflowcontrol.go
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,8 @@ type raftAdmissionMetaKey struct{}
// ContextWithMeta returns a Context wrapping the supplied raft admission meta,
// if any.
//
// TODO(irfansharif): This causes a heap allocation. Revisit as part of #95563.
// TODO(irfansharif,aaditya): This causes a heap allocation. Revisit as part of
// #104154.
func ContextWithMeta(ctx context.Context, meta *kvflowcontrolpb.RaftAdmissionMeta) context.Context {
if meta != nil {
ctx = context.WithValue(ctx, raftAdmissionMetaKey{}, meta)
Expand Down
Loading

0 comments on commit 2b6e80e

Please sign in to comment.