Prevent repeated resumes of a resource by remembering its resumed flag

[nolar]

Issue : #113 #223 #214

Description

This PR does two things:

First, it adds a per-resource in-memory container, which is remembered for the whole lifecycle of an operator. This can lead to more memory consumption on big clusters.

It is different from the persistent storage of the object's data directly on the object's status, as the values stored in the in-memory container are specific to that operator process only, and sometimes not serialisable (e.g. threads, tasks, asyncio events, locks, etc).

Second, it remember every object's resuming status in-memory. Once resumed, the object is never resumed again. Otherwise (i.e. currently), the resuming happens on every reconnection of the API watch-stream (partially remedied by #229, but not fully).

More on that, currently (i.e. before fixing), if there are multiple resume handlers or retries or sub-handlers, only the first attempt will be executed. Also, if the on-resume handlers go after the on-create/on-update handlers, they are ignored. All other attempts after the initial listing will not be interpreted as "initial" (due to event's type not being None anymore), thus not detected as resuming.

With this PR, the resume handlers are included into the selection until all of them succeed at least one — even for the regular watch-events with ['type']!=None (i.e. after patching from the previous attempts).

---

Such approach should improve the following for the @on.resume handlers:

1. Executed only once per operator process life time (in case of success).
2. Retried until timeout or retries limit (in case of failures).
3. Any order of handlers supported (e.g. on-resume handlers after the on-create handlers).
4. Multiple on-resume handlers supported.
5. Sub-handlers in the on-resume handlers supported.

Types of Changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor/improvements

Review

List of tasks the reviewer must do to review the PR

• Tests
• Documentation

[zincr]

🤖 zincr found 0 problems , 0 warnings

✅ Large Commits
✅ Approvals
✅ Specification
✅ Dependency Licensing

[zincr]

🤖 zincr found 1 problem , 0 warnings

❌ Approvals
✅ Large Commits
✅ Specification
✅ Dependency Licensing

Details on how to resolve are provided below

---

Approvals

All proposed changes must be reviewed by project maintainers before they can be merged

Not enough people have approved this pull request - please ensure that 1 additional user, who have not contributed to this pull request approve the changes.

• ✅ Approved by PR author @nolar
• ❌ 1 additional approval needed
   

[zincr]

🤖 zincr found 1 problem , 0 warnings

❌ Approvals
✅ Large Commits
✅ Specification
✅ Dependency Licensing

Details on how to resolve are provided below

---

Approvals

All proposed changes must be reviewed by project maintainers before they can be merged

Not enough people have approved this pull request - please ensure that 1 additional user, who have not contributed to this pull request approve the changes.

• ✅ Approved by PR author @nolar
• ❌ 1 additional approval needed