Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
db: introduce CompactionScheduler and integrate with DB
CompactionScheduler is an interface that encompasses (a) our current
compaction scheduling behavior, (b) compaction scheduling in a multi-store
setting that adds a per-node limit in addition to the per-store limit, and
prioritizes across stores, (c) compaction scheduling that includes (b) plus
is aware of resource usage and can prioritize across stores and across
other long-lived work in addition to compactions (e.g. range snapshot
reception).
CompactionScheduler calls into DB and the DB calls into the
CompactionScheduler. This requires some care in specification of the
synchronization expectations, to avoid deadlock. For the most part, the
complexity is borne by the CompactionScheduler -- see the code comments
for details.
ConcurrencyLimitScheduler is an implementation for (a), and is paired with
a single DB. It has no knowledge of delete-only compactions, so we have
redefined the meaning of Options.MaxConcurrentCompactions, as discussed
in the code comment.
CompactionScheduler has some related interfaces/structs:
- CompactionGrantHandle is used to report the start and end of the
compaction, and frequently report the written bytes, and CPU consumption.
In the implementation of CompactionGrantHandle provided by CockroachDB's
AC component, the CPU consumption will use the grunning package.
- WaitingCompaction is a struct used to prioritize the DB's compaction
relative to other long-lived work (including compactions by other DBs).
makeWaitingCompaction is a helper that constructs this struct.
For integrating the CompactionScheduler with DB, there are a number of
changes:
- The entry paths to ask to schedule a compaction are reduced to 1, by
removing DB.maybeScheduleCompactionPicker. The only path is
DB.maybeScheduleCompaction.
- versionSet.{curCompactionConcurrency,pickedCompactionCache} are added
to satisfy the interface expected by CompactionScheduler. Specifically,
pickedCompactionCache allows us to safely cache a pickedCompaction
that cannot be run. There is some commentary on the worst-case waste
in compaction picking -- with the default ConcurrencyLimitScheduler
on average there should be no wastage.
- versionSet.curCompactionConcurrency and DB.mu.compact.manualLen are two
atomic variables introduced to implement DB.GetAllowedWithoutPermission,
which allows the DB to adjust what minimum compaction concurrency it
desires based on the backlog of automatic and manual compactions. The
encoded logic is meant to be equivalent to our current logic.
The CompactionSlot and CompactionLimiter introduced in a recent PR are
deleted. CompactionGrantHandle is analogous to CompactionSlot, and allows for
measuring of CPU usage since the implementation of CompactionScheduler in AC
will explicitly monitor usage and capacity. CompactionScheduler is analogous to
CompactionLimiter. CompactionLimiter had a non-queueing interface in
that it never called into the DB. This worked since the only events that
allowed another compaction to run were also ones that caused another
call to maybeScheduleCompaction. This is not true when a
CompactionScheduler is scheduling across multiple DBs, or managing a
compaction and other long-lived work (snapshot reception), since something
unrelated to the DB can cause resources to become available to run a
compaction.
There is a partial implementation of a resource aware scheduler in
https://github.com/sumeerbhola/cockroach/tree/long_lived_granter/pkg/util/admission/admit_long.
Informs #3813, cockroachdb/cockroach#74697, #1329- Loading branch information
1 parent
35df196
commit a8b60cf