building and testing sidecar containers

[pohly]

For new developers it would be useful to document how they can build and test CSI. This could include:

• building and pushing container images for driver-registrar/hostpath (aka drivers)/external-attacher/external-provisioner
• running the kubernetes/test/e2e/storage/csi_volumes.go test for hostpath

[pohly]

Right now, building container images is inconsistent. Some makefiles have a "push" target, some don't. Some build "canary" as version, others the latest stable release.

My proposal is to have the following in all four makefiles:

REGISTRY_NAME=quay.io/k8scsi
IMAGE_NAME=<actual image name>
IMAGE_VERSION=canary
IMAGE_TAG=$(REGISTRY_NAME)/$(IMAGE_NAME):$(IMAGE_VERSION)

container: ....
	docker build -t $(IMAGE_TAG) .

push: container
	docker push $(IMAGE_TAG)

Then one can do:

for i in driver-registrar/ external-attacher/ external-provisioner/ drivers/; do make -C $i REGISTRY_NAME=localhost:5000 push; done

The rationale for only supporting building/pushing of the hostpath container in drivers is that long-term that should be the only image in the repo. Already now it is the only image that gets built by make.

I have that working and can create PRs once there is some positive feedback about the idea (no need to discuss it in four places).

I also have a change ready for kubernetes/test/e2e/storage which makes it possible to change the image version and/or registry:

go run hack/e2e.go -- --provider=local --test --test_args="--ginkgo.focus=CSI.plugin.test.using.CSI.driver..hostPath -csiImageVersion=canary -csiImageRegistry=localhost:5000"

This can be used to test the canary images on quay.io/k8scsi before tagging them as a new release.

[lpabon]

@pohly The drivers repo is going to be dismantled. See kubernetes-retired/drivers#81. AFAIK, the side-car containers are all pushing canary containers into Quay.io.

I guess I would need to understand more why there would be a need to build all at the same time. But I am open to discussion.

Lastly, are the e2e tests using local cluster?

[pohly]

@pohly The drivers repo is going to be dismantled. See kubernetes-retired/drivers#81.

Yes. That's why I think it is okay to focus on building just the hostpath container image in the "drivers" repo, because that is going to remain.

AFAIK, the side-car containers are all pushing canary containers into Quay.io.

Indeed, they do now. My revision of driver-registrar was a bit old and still had IMAGE_VERSION=v0.2.0.

I guess I would need to understand more why there would be a need to build all at the same time.

Not at the same time, just the same way.

Lastly, are the e2e tests using local cluster?

They can use a local cluster or something remote. I myself run the test with a cluster brought up with local-up-cluster.sh.

[pohly]

I have created four PRs, one for each of the repos, where I add the same change: enabling make REGISTRY_NAME=localhost:5000 push

[pohly]

I think the best solution for this issue is to proceed with the unification of the different repos, see https://docs.google.com/document/d/1xC0Cm11WTebT9m_9yZbfzud24GJRxlp55gdR5QaMHmM/edit

[lpabon]

Right now this process is on hold until K8S 1.11 is released.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[msau42]

are there more things to do for this?

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.