Update the EphemeralVolumeClaim story-telling #97

nolar · 2019-06-02T12:37:00Z

This is a work in progress. Preview, but do not review.

Issue : #12

Description

The docs were showing some basic Kopf features, but they were not actually tested to work, both syntactically, and functionally.

This PR fixes all the issues with the EphemeralVolumeClaim in the docs tutorial (step by step example of developing and operator).

The actual operator is now implemented and manually tested here: https://github.com/nolar/ephemeral-volume-claims — all code snippets are taken from this repo.

A few sentences describing the overall goals of the pull request's
commits.

Types of Changes

Bug fix (non-breaking change which fixes an issue)
Documentation / non-code

Tasks

EVC operator itself:
- Spy on pods that use the claims (uses Add the silent spy-handlers for the raw events, no persistence #86, blocked by Support for built-in resources (pods, jobs, etc) #84 )
- Track the pods-claims usage.
- Delete the EphemeralVolumeClaim+PersistentVolumeClaim after expiration/grace periods.
- Test it to actually work (manually).
Docs:
- Fix page: resizing with @on.update handler: Minikube cannot resize PVCs/PVs.
- New page: pod spying, @on.resume handler, countdown tasks in background.
- Reference to the implementation.

zincr · 2019-06-02T12:37:11Z

🤖 zincr found 0 problems , 0 warnings

✅ Large Commits
✅ Approvals
✅ Specification
✅ Dependency Licensing

zincr · 2019-06-02T12:37:12Z

🤖 zincr found 1 problem , 0 warnings

❌ Approvals
✅ Large Commits
✅ Specification
✅ Dependency Licensing

Details on how to resolve are provided below

Approvals

All proposed changes must be reviewed by project maintainers before they can be merged

Not enough people have approved this pull request - please ensure that 1 additional user, who have not contributed to this pull request approve the changes.

✅ Approved by PR author @nolar
❌ 1 additional approval needed

yashbhutwala

I had to struggle through the docs, and I was about to submit a PR, then I saw this :-). It seems this is good enough to update the live docs, maybe merge it soon? So, someone else doesn't suffer as me haha

yashbhutwala · 2019-06-19T00:37:47Z

docs/walkthrough/updates.rst

+
+.. warning::
+    Kubernetes & ``kubectl`` improperly show the capacity of PVCs:
+    it remains the same (1G) event after the change.


event --> even

yashbhutwala · 2019-06-19T00:41:32Z

docs/walkthrough/updates.rst


    @kopf.on.create('zalando.org', 'v1', 'ephemeralvolumeclaims')
-    def create_fn(spec, meta, namespace, logger, **kwargs):
+    def create_fn(body, spec, meta, namespace, logger, **kwargs):


don't need body here. delete kopf.adopt(data, owner=body) because you only show it later in cascaded deletion.

yashbhutwala · 2019-06-19T00:42:35Z

docs/walkthrough/updates.rst


        name = meta.get('name')
        size = spec.get('size')
        if not size:
            raise kopf.HandlerFatalError(f"Size must be set. Got {size!r}.")

-        path = os.path.join(os.path.dirname(__file__), 'pvc-tpl.yaml')
+        path = os.path.join(os.path.dirname(__file__), 'pvc.yaml')
        tmpl = open(path, 'rt').read()
        text = tmpl.format(size=size, name=name)
        data = yaml.load(text)


yaml.load(text) --> yaml.load(text, Loader=yaml.FullLoader) because https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation

Oh yes, I saw that warning already few times. Will fix that too.

nolar · 2019-06-19T15:27:55Z

@yashbhutwala Sorry for the yet incomplete docs. This was not intended to make anyone struggling :-)

This PR assumes that #110 is finished — which is "almost there", close to the end (also needs #109 before).

Originally, I intended to also finish the actual ephemeral-volume-claim-operator with all the features.

But you are right, the docs consistency is important — especially since Kopf got some traction suddenly. I will merge this PR as soon as #109+#110 are ready, and will continue adding the missing docs separately.

yashbhutwala · 2019-06-19T15:33:34Z

No worries, @nolar. Let me know if you need help 😄

piratos

found this issue while running through the readthedocs walk-through

piratos · 2019-10-05T18:49:36Z

docs/walkthrough/deletion.rst


        name = meta.get('name')
        size = spec.get('size')
        if not size:
            raise kopf.HandlerFatalError(f"Size must be set. Got {size!r}.")

-        path = os.path.join(os.path.dirname(__file__), 'pvc-tpl.yaml')
+        path = os.path.join(os.path.dirname(__file__), 'pvc.yaml')


os is not previously imported

asmacdo

I made it through the tutorial pretty smoothly with this PR, thank you!

asmacdo · 2019-12-06T19:44:34Z

docs/walkthrough/creation.rst

@@ -53,6 +53,7 @@ We will use the official Kubernetes client library:
    :linenos:
    :caption: ephemeral.py

+    import os
    import kopf
    import kubernetes


Nitpick: IMO it might be worth a note pip install kubernetes.

asmacdo · 2019-12-06T19:50:49Z

docs/walkthrough/updates.rst

@@ -74,11 +74,11 @@ and patches the PVC with the new size from the EVC::
    @kopf.on.update('zalando.org', 'v1', 'ephemeralvolumeclaims')
    def update_fn(spec, status, namespace, logger, **kwargs):

-        size = spec.get('create_fn', {}).get('size', None)
+        size = status.get('size', None)


This didn't work for me. I think it should be spec.get('size', None)

Indeed. Solved.

nolar · 2020-01-28T11:42:37Z

This PR is waiting for so long, that I have lost track of what was left. So, let's merge it as is. The tutorial story has to be redesigned/reviewed/rewritten anyway — especially with the per-pod ephemeral inline volumes coming in Kubernetes 1.16 by default.

nolar · 2020-02-13T20:08:30Z

fixes #238

nolar added the documentation Documentation improvements label Jun 2, 2019

nolar added this to the 1.0 milestone Jun 2, 2019

This was referenced Jun 10, 2019

Automated OLM generation and verification #106

Open

Switch to pykube-ng to handle arbitrary resources (pods, jobs, etc) #110

Merged

yashbhutwala suggested changes Jun 19, 2019

View reviewed changes

piratos suggested changes Oct 5, 2019

View reviewed changes

nolar force-pushed the 12-evc-story branch from c4bcdb0 to 0bc9b44 Compare November 13, 2019 14:19

asmacdo reviewed Dec 6, 2019

View reviewed changes

Sergey Vasilyev added 6 commits January 28, 2020 12:30

Fix the alignment of code samples, remove an extra space on each line

e9d58db

Fix the code issues in the EphemeralVolumeClaim operator in docs

3a90cef

Decrease the size of the volumes - for easier local experimentation

f14ff1e

Show the actual output of some commands and operator's actions

9d750c5

Warn that Minikube cannot handle the PVC/PV resizes properly

2dacc44

Refer to the full implementation of EphemeralVolumeClaim operator

8c54ea9

nolar force-pushed the 12-evc-story branch from 0bc9b44 to 8c54ea9 Compare January 28, 2020 11:34

nolar added 2 commits January 28, 2020 12:35

Mention that kubernetes must be installed before use

5d8466e

Fix the imports (as per PR comments)

cfd8296

nolar marked this pull request as ready for review January 28, 2020 11:37

nolar changed the title ~~Update the EphemeralVolumeClaim story-telling in the docs/tutorial~~ Update the EphemeralVolumeClaim story-telling Jan 28, 2020

nolar requested a review from aweller January 30, 2020 15:05

aweller approved these changes Jan 30, 2020

View reviewed changes

nolar merged commit 6eb5a5a into zalando-incubator:master Jan 30, 2020

nolar deleted the 12-evc-story branch January 30, 2020 18:09

nolar mentioned this pull request Feb 13, 2020

Error in kopf/updates.rst #238

Closed

kopf-archiver bot mentioned this pull request Aug 19, 2020

[PR] Update the EphemeralVolumeClaim story-telling nolar/kopf#97

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the EphemeralVolumeClaim story-telling #97

Update the EphemeralVolumeClaim story-telling #97

nolar commented Jun 2, 2019

zincr bot commented Jun 2, 2019 •

edited

Loading

zincr bot commented Jun 2, 2019

yashbhutwala left a comment

yashbhutwala Jun 19, 2019

yashbhutwala Jun 19, 2019

yashbhutwala Jun 19, 2019

nolar Jun 19, 2019

nolar commented Jun 19, 2019

yashbhutwala commented Jun 19, 2019

piratos left a comment

piratos Oct 5, 2019

asmacdo left a comment

asmacdo Dec 6, 2019

asmacdo Dec 6, 2019

nolar Jan 28, 2020

nolar commented Jan 28, 2020

nolar commented Feb 13, 2020

Update the EphemeralVolumeClaim story-telling #97

Update the EphemeralVolumeClaim story-telling #97

Conversation

nolar commented Jun 2, 2019

Description

Types of Changes

Tasks

zincr bot commented Jun 2, 2019 • edited Loading

🤖 zincr found 0 problems , 0 warnings

zincr bot commented Jun 2, 2019

🤖 zincr found 1 problem , 0 warnings

Approvals

yashbhutwala left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nolar commented Jun 19, 2019

yashbhutwala commented Jun 19, 2019

piratos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asmacdo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nolar commented Jan 28, 2020

nolar commented Feb 13, 2020

zincr bot commented Jun 2, 2019 •

edited

Loading