Skip to content

Commit

Permalink
doc: UML diagram for volume creation and deletion
Browse files Browse the repository at this point in the history
While this information is available in various design documents, users
of the external-provisioner benefit from getting a summary directly in
the documentation of external-provisioner. This is a good opportunity
to call our expected behavior of the CSI driver.
  • Loading branch information
pohly committed Nov 3, 2020
1 parent 3582478 commit 1c8273e
Show file tree
Hide file tree
Showing 5 changed files with 73 additions and 0 deletions.
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ CMDS=csi-provisioner
all: build

include release-tools/build.make
include doc/doc.make
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,34 @@ See the [storage capacity section](#capacity-support) below for details.

* All glog / klog arguments are supported, such as `-v <log level>` or `-alsologtostderr`.

### Design

External-provisioner interacts with Kubernetes by watching PVCs and
PVs and implementing the [external provisioner
protocol](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#provisioning-and-deleting). The
following diagram illustrates that, starting with the creation of a
PVC by a user and ending with the removal of all resources.

![](doc/provisioning.png)

When an object is marked for deletion, actual deletion is delayed
until the controller checks the necessary preconditions for deletion
and removes its finalizer.

Failures are handled by retrying, which is only shown here for
`CreateVolume` calls because it is a bit special: for late binding, a
CSI driver can trigger the re-scheduling of a pod by returning the
special `ResourceExhausted` gRPC status code.

Note that a timed-out `CreateVolume` call does not change any state
and will simply get repeated. If the CSI driver has completed volume
creation in the meantime, it must recognize the repeated call and
return information about it ("idempotency"). All gRPC calls have to be
idempotent.

How the selected node is communicated to the CSI driver is explained
in the next section.

### Topology support
When `Topology` feature is enabled and the driver specifies `VOLUME_ACCESSIBILITY_CONSTRAINTS` in its plugin capabilities, external-provisioner prepares `CreateVolumeRequest.AccessibilityRequirements` while calling `Controller.CreateVolume`. The driver has to consider these topology constraints while creating the volume. Below table shows how these `AccessibilityRequirements` are prepared:

Expand Down
13 changes: 13 additions & 0 deletions doc/doc.make
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# We use the http://plantuml.com/plantuml server to generate
# images. That way nothing needs to be installed besides Go.
# The client tool has no license, so we can't vendor it.
# Instead we "go get" it if (and only if) needed.
DOC_PLANTUML_GO = $(shell go env GOPATH)/bin/plantuml-go

%.png: %.puml $(DOC_PLANTUML_GO)
$(DOC_PLANTUML_GO) -format png $<

# Builds the binary in GOPATH/bin. Changing into / first avoids
# modifying the project's go.mod file.
$(DOC_PLANTUML_GO):
cd / && go get github.com/acarlson99/plantuml-go
Binary file added doc/provisioning.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
31 changes: 31 additions & 0 deletions doc/provisioning.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
@startuml
(*) -->[user creates PVC] "PVC exists"
-->[PV controller checks storage class and existing PVs, sets the\nvolume.beta.kubernetes.io/storage-provisioner annotation] "PVC ready for provisioning" as PVC_READY
--> ==B1==

==B1== -->[immediate binding] "CreateVolume for all nodes" as CREATE_VOLUME_ALL
-->[failure] "PVC_READY"
CREATE_VOLUME_ALL -->[success] ==B2==

==B1== -->[late binding] "wait for pod" as POD
-->[scheduler tentatively chooses a node for a pod\nwhich uses the volume and sets the\nvolume.beta.kubernetes.io/selected-node annotation] "PVC with selected node ready for provisioning" as PVC_READY_SELECTED
--> "CreateVolume for selected node" as CREATE_VOLUME_SELECTED
-->[ResourceExhausted:\nexternal-provisioner unsets the\nselected-node annotation] "POD"
CREATE_VOLUME_SELECTED -->[failure] "PVC_READY_SELECTED"
CREATE_VOLUME_SELECTED -->[success] ==B2==

==B2== --> "volume exists" as VOLUME_EXISTS
-->[external-provisioner creates PV] "PV ready for binding"
-->[PV controller sets 'PV.Status.Phase: Bound'] "PV bound"
-->[PV controller sets\npv.kubernetes.io/bind-completed: yes\npv.kubernetes.io/bound-by-controller: yes\n'PVC.Spec.VolumeName'] "PVC almost bound"
-->[PV controller sets 'PVC.Status.Phase: Bound'\nand 'PVC.Status.Capacity'] "PVC bound"
-->[user deletes PVC] "PVC marked for deletion"
-->[PVC protection controller determines that\nPVC is not in use] "PVC deleted, PV orphaned" as VOLUME_READY_FOR_DELETION
--> "DeleteVolume"
DeleteVolume -->[failure] VOLUME_READY_FOR_DELETION
DeleteVolume -->[success] "volume deleted"
-->[external-provisioner set 'PV.Status.Phase: Released'] "PV ready for deletion" as PV_READY_FOR_DELETION
-->[external-provisioner deletes PV] "PV marked for deletion"
-->[PV controller determines that\nPV has no volume] "PV deleted"
--> (*)
@enduml

0 comments on commit 1c8273e

Please sign in to comment.