[PR] Ensure that all handlers get the latest state (eventually)

[kopf-archiver]

A pull request by nolar at 2019-08-02 01:36:20+00:00
Original URL: zalando-incubator/kopf#163
Merged by nolar at 2019-08-08 14:54:14+00:00

Keep track of which state was handled by which handler, and re-execute it if the state is changed during the handling cycle.

Issue : replays #9, indirectly #161 and #150

Description

In #9, there was an attempt to make the handler's status persistent. As a side-effect, it also could detect the state changes during the handling cycle. The persistent progress was not needed or not much useful. However, the change detection is needed again. This PR partially replays the same solution.

Currently, if an object is changed during the long handling cycle (which can include multiple handlers and multiple retries — i.e. for hours sometimes), these changes will not be noticed in some handlers. The "last-seen" state will be stored as it was at the moment of the last handler, and only the changes relative to that state will be handled since then.

As a result, if some handler happened to run before the change was introduced, i.e. it has got an old state, that handler would never know about the new state.

With this PR, all update & deletion handlers handlers will track the handled state of the objects individually (as a short hash-digest). If it changes during the handling cycle, those handlers will be re-executed with the new state.

This guarantees that every handler will see the latest actual state of the object — the closest to the concept of reconciliation.

For creation handlers, the behaviour is slightly different. In case of changes during the creation cycle, the creation handlers will be continued as normally (not re-executed). And only after the full creation cycle is done, an update cycle will start with the changes relatively to the first (not last!) creation handler, i.e. when the object was seen in the first time.

This should prevent spawning duplicate children objects, which is usually done in the creation handlers. The update handlers should be designed idempotent anyway, so it is safe to shift the repeated handling of the same state from the creation handlers to the update handlers.

Types of Changes

• New feature (non-breaking change which adds functionality)

Review

• Tests
• Documentation

---

Commented by nolar at 2019-08-08 12:29:23+00:00
 

Rebased on master with amendments for the new codebase & features.