Field filters for all resource handlers

[nolar]

Implement field & value filters for all resource-changing handlers: @kopf.on.resume/create/update/delete.

It was a long-needed feature to replace field handlers (@kopf.on.field) with more clear, explicit, and less ambiguous syntax. It was drafted in Jan'2020, but didn't get into the main codebase due to some semantical complications. Over the past year, the complications were resolved, so it seems easy to implement this feature now.

If field= is set on any handler, the handler is triggered only if the field is present in the object, and old/new/diff kwargs are reduced to that field instead of the whole body; the usually body & co kwargs still point to the full body, as expected.

@kopf.on.create(..., field='spec.field')
def fn(new, **_):
  print(f"{new!r}")

As with other filters (labels, annotations), either specific values are accepted and checked for equality, or kopf.ABSENT/kopf.PRESENT markers are accepted, or callbacks that accept a positional value and all typical kwargs, and return a boolean for whether it matches or not.

@kopf.on.create(..., field='spec.field', value='value')
@kopf.on.create(..., field='spec.field', value=kopf.PRESENT)
@kopf.on.create(..., field='spec.field', value=kopf.ABSENT)
@kopf.on.create(..., field='spec.field', value=lambda val, **_: 'a' in val)
@kopf.on.create(..., field='spec.field', value=kopf.any_([fn1, fn2]))
@kopf.on.create(..., field='spec.field', value=kopf.all_([fn1, fn2]))
@kopf.on.create(..., field='spec.field', value=kopf.none_([fn1, fn2]))
def fn(**_): pass

The default behaviour depends on what kind of a filter/handler it is:

For single-value filters (creation/resuming/deletion handlers), by default (i.e. value=None), the field must be present, and it is equivalent to value=kopf.PRESENT:

@kopf.on.create(..., field='spec.field')
def fn(**_): pass

For the update & on-field handlers, additional criteria can be used: old & new. By default, it is only triggered if the value is changed (including the case when it is changed from anything to None — i.e. the field is removed):

@kopf.on.update(..., field='spec.field')  # if changed (incl. if removed)
@kopf.on.update(..., field='spec.field', old='x', new='y')  # only this specific change
@kopf.on.update(..., field='spec.field', old=kopf.ABSENT, new=kopf.PRESENT)  # the field was set
@kopf.on.update(..., field='spec.field', old=kopf.PRESENT, new=kopf.ABSENT)  # the field was unset
def fn(**_): pass

Using both value= & old=/new= is prohibited, as it is confusing. It is either-either. Essentially, new is the same as value.

---

Some unusual tricks are now possible — as side-effects, not as the main goal of this change:

Trigger a handler only if the required field is NOT defined — the opposite of the default behaviour. The value of new will be None:

@kopf.on.create(..., field='spec.nofield', value=kopf.ABSENT)
def fn(**_): pass

---

@kopf.on.field handler are now discouraged — but not yet deprecated. They will most likely remain for some long time for backward compatibility. A field handler like this:

@kopf.on.field(..., field='spec.field')
def fn(old, new, **_):
    pass

Is equivalent to this:

@kopf.on.create(..., field='spec.field')
@kopf.on.update(..., field='spec.field')
def fn(old, new, **_):
    pass

Note: but not for deletion & resuming, as nothing changes in those occasions, and on-field is only triggered when something is changed.

---

The field-value (but not old/new) filtering logic can also be applied to all other resource handlers:

import kopf

@kopf.on.event(..., field='spec.field', value='value')
def event_fn(**_):
    pass

@kopf.timer(..., field='spec.field', value='value', interval=1)
def timer_fn(**_):
    pass

@kopf.daemon(..., field='spec.field', value='value')
def event_fn(stopped, **_):
    stopped.wait(1000)

Their semantics is the same as the labels/annotations/callback filtering: the handler is invoked and the daemon is running as long as the criterion is satisfied. Once the resource changes so that it does not satisfy the criterion, the handlers are not invoked, and the daemon is stopped.

---

Remaining TODOs:

• Test things manually and verify that the behaviour is consistent and intuitive.
• Raise field&value (but not old&new) filters to all resource handlers: on.event, daemon, timer.
• Performance optimisations for dict lookups.
• Unit-tests.
• An example (or extend the example 05-handlers?).
• Documentation (preview: https://kopf.readthedocs.io/en/field-filters/filters/)

Related: #571 #566

[lgtm-com]

This pull request introduces 1 alert when merging 2137366 into 8023ea4 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging af33857 into 8023ea4 - view on LGTM.com

new alerts:

• 1 for Unused import

[nolar]

Documentation preview: https://kopf.readthedocs.io/en/field-filters/filters/

[nolar]

Tested manually. The field-filters work as expected, it seems. Even for sub-handlers with this setup (impractical, but possible):

import kopf


@kopf.on.create('zalando.org', 'v1', 'kopfexamples')
async def created_with_field(**kwargs):
    print(f'CREATED')
    for i in range(3):

        @kopf.on.this(field='spec.field', value='value', id=f'a{i}')
        def a_sub(new, i=i, **kwargs):
            print(f'A-SUB {i} with field: {new}')

        @kopf.on.this(field='spec.field', value=kopf.PRESENT, id=f'b{i}')
        def b_sub(new, i=i, **kwargs):
            print(f'B-SUB {i} with field: {new}')


@kopf.on.update('zalando.org', 'v1', 'kopfexamples')
async def update_with_field(**kwargs):
    print(f'UPDATED')
    for i in range(3):

        @kopf.on.this(field='spec.field', old=i-1, new=i, id=f'c{i}')
        def c_sub(new, i=i, **kwargs):
            print(f'C-SUB {i} with field: {new}')

And then doing:

kubectl apply -f examples/obj.yaml
kubectl patch -f examples/obj.yaml --type merge -p '{"spec": {"field": -1}}'
kubectl patch -f examples/obj.yaml --type merge -p '{"spec": {"field": 0}}'
kubectl patch -f examples/obj.yaml --type merge -p '{"spec": {"field": 1}}'
kubectl patch -f examples/obj.yaml --type merge -p '{"spec": {"field": 2}}'
kubectl patch -f examples/obj.yaml --type merge -p '{"spec": {"field": 0}}'  # no reaction