[PR] Daemons & timers for background processing and reconciliation

[kopf-archiver]

A pull request by nolar at 2020-03-12 09:10:51+00:00
Original URL: zalando-incubator/kopf#330
Merged by nolar at 2020-04-01 10:17:06+00:00

What do these changes do?

Add background daemons & timers for resources.

Description

Previously, the handlers were event-driven — i.e. when something changes on the resource. If nothing changes, no handlers are called. To have reconciliation or other activities happening during the lifetime of a resource, the developers had to do their own task/thread orchestration.

With this PR, the task/thread orchestration is generalised to a new framework feature to perform background reconciliation with external systems, locally managed applications, or even local children K8s resources not the event-driven way:

import kopf

@kopf.daemon('zalando.org', 'v1', 'kopfexamples', initial_backoff=5.0, cancellation_backoff=10.0)
async def background_async(spec, logger, **_):
    while True:  # async handlers will be cancelled: instantly or with a graceful delay.
        logger.info(f"Ping from an async daemon: field={spec['field']!r}")
        await asyncio.sleep(2.5)

@kopf.timer('zalando.org', 'v1', 'kopfexamples', idle=15, interval=2, sharp=True)
def every_few_seconds_sync(spec, logger, **_):
    logger.info(f"Ping from a sync timer: field={spec['field']!r}")

The daemons run as long as the resource exists. They are stopped/cancelled when the resource is deleted (or when the operator exits/restarts). The daemons can be automatically restarted or left dead if exited (restart_backoff).

The timers are also triggered as long as the resource exists, but by schedule. They can be postponed by few seconds since the last change (idle), or since the resource creation / operator startup (initial_backoff), can have sharp or soft intervals (see the docs), etc.

For more examples & options, see /examples/ folder and the doc updates in this PR/branch.

Issues/PRs

Issues: closes #19

Related: #150 #271 #317 #122

Type of changes

• New feature (non-breaking change which adds functionality)

Checklist

• The code addresses only the mentioned problem, and this problem only
• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes
• If you provide code modification, please add yourself to CONTRIBUTORS.txt

---

WARNING: This PR is a preview. It is runnable and usable, but few minor things are not finished yet:

• Tests
  • For the stopper synchronization primitive.
  • For daemon existing reason signalling (DaemonStopperReason).
  • For daemon & timer decorators.
  • For async daemon cancellation.
  • For sync daemon cancellation.
  • For timer guarding task cancellation.
• A sync-daemon did not exit on operator stopping in a try-run (it did exit before, something is broken? — check & fix.).

---

Commented by nolar at 2020-03-31 08:32:03+00:00
 

The tests are rudimentary, only for the smoke-testing of daemons & timers. This PR took already too much time, and is too big.

The low-level primitives are tested indirectly through the existing tests, and manually by running examples with Minikube (including the examples 14 & 15 for daemons & timers).

The framework itself will be tested for backward compatibility by deployed a release candidate to our own clusters and utilising the new features (incl. daemons & timers).

More detailed tests for all little aspects will be added later as separate PRs — with lower priority (time-wise).