Merge pull request #298 from fluxcd/release/1.0.0-rc9

Release 1.0.0-rc9
fluxcd · Feb 13, 2020 · f7e4871 · f7e4871
2 parents c6ffc8f + 35732d8
commit f7e4871
Show file tree

Hide file tree

Showing 71 changed files with 1,641 additions and 627 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -1,4 +1,16 @@
-version: 2
+version: 2.1
+commands:
+  e2e_tests:
+    parameters:
+      helm_version:
+        type: string
+    steps:
+      - run:
+          name: End-to-end Helm <<parameters.helm_version>>
+          command: make e2e
+          environment:
+            E2E_KIND_CLUSTER_NUM: 4
+            HELM_VERSION: <<parameters.helm_version>>
 jobs:
   build:
     working_directory: /home/circleci/go/src/github.com/fluxcd/helm-operator
@@ -55,16 +67,10 @@ jobs:
       - run: make test TEST_FLAGS="-race -timeout 5m"
       - run: make all
       - run: make test-docs
-      - run:
-          name: End-to-end Helm v2
-          command: E2E_KIND_CLUSTER_NUM=4 make e2e
-          environment:
-            HELM_VERSION: v2
-      - run:
-          name: End-to-end Helm v3
-          command: E2E_KIND_CLUSTER_NUM=4 make e2e
-          environment:
-            HELM_VERSION: v3
+      - e2e_tests:
+          helm_version: v2
+      - e2e_tests:
+          helm_version: v3
       - save_cache:
           key: cache-{{ checksum "Makefile" }}
           paths:

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,86 @@
+## 1.0.0-rc9 (2020-02-13)
+
+> **Notice:** upgrading to this version from `<=0.10.x` by just
+> updating your Helm Operator image tag is not possible as the
+> CRD domain and version have changed. An upgrade guide can be
+> found [here](./docs/guides/upgrading-to-ga.md).
+
+> **Notice:**  due to the multiple added fields, you need to
+> re-apply the `HelmRelease` CRD.
+
+### Bug fixes
+
+ - release: propagate all configured release flags to dry-run upgrade
+   [fluxcd/helm-operator#250][#250]
+ - chartsync: honour the configured default Git ref when reconciling
+   charts source
+   [fluxcd/helm-operator#253][#253]
+ - release: disable atomic flag for Helm chart installation
+   [fluxcd/helm-operator#256][#256]
+ - apis: correct JSON namespace tag for key selectors
+   [fluxcd/helm-operator#262][#262]
+ - helm/v3: support upgrades of releases with nested `HelmRelease`
+   resources (using a patched Helm `3.0.3` release)
+   [fluxcd/helm-operator#292][#292]
+
+### Enhancements
+
+ - release: support retrying rollbacks
+   [fluxcd/helm-operator#252][#252]
+ - helm: support downloader plugins
+   [fluxcd/helm-operator#263][#263]
+ - helm/v3: support skipping CRD installation using `.spec.skipCRDs`
+   [fluxcd/helm-operator#282][#282]
+ - helm/v3: enrich Helm logger with release name and namespace
+   metadata
+   [fluxcd/helm-operator#291][#291]
+
+### Maintenance and documentation
+
+ - e2e: use podinfo's `--unready` to make a release fail
+   [fluxcd/helm-operator#258][#258]
+ - Pkg: update Helm 3 to `3.0.3`
+   fluxcd/helm-operator{[#260][], [#292][]}
+ - build: include `bash` and `curl` in image
+   [fluxcd/helm-operator#276][#267]
+ - build: make sure we test all the local modules
+   [fluxcd/helm-operator#269][#269]
+ - build: add `generate-codegen` target to Makefile
+   [fluxcd/helm-operator#289][#289]
+ - e2e: install Tiller in operator namespace for more reliable cleanup
+   [fluxcd/helm-operator#290][#290]
+ - e2e: do not create kind clusters in parallel
+   [fluxcd/helm-operator#290][#290]
+ - docs: document usage of Helm downloader plugins
+   [fluxcd/helm-operator#295][#295]
+ - docs: highlight standalone usage in `README.md`
+   [fluxcd/helm-operator#296][#296]
+
+### Thanks
+
+Thanks to @sa-spag, @stefanprodan, @mcharriere, @GODBS, @derrickburns,
+@autarchprinceps, @stefanseditch, @infinitydon, @cbenjemaa, @sayboras,
+@2opremio, @hiddeco, and others for their contributions to this
+release, feedback, and reporting issues.
+
+[#250]: https://github.com/fluxcd/helm-operator/pull/250
+[#252]: https://github.com/fluxcd/helm-operator/pull/252
+[#253]: https://github.com/fluxcd/helm-operator/pull/253
+[#256]: https://github.com/fluxcd/helm-operator/pull/256
+[#258]: https://github.com/fluxcd/helm-operator/pull/258
+[#260]: https://github.com/fluxcd/helm-operator/pull/260
+[#262]: https://github.com/fluxcd/helm-operator/pull/262
+[#263]: https://github.com/fluxcd/helm-operator/pull/263
+[#267]: https://github.com/fluxcd/helm-operator/pull/267
+[#269]: https://github.com/fluxcd/helm-operator/pull/269
+[#282]: https://github.com/fluxcd/helm-operator/pull/282
+[#289]: https://github.com/fluxcd/helm-operator/pull/289
+[#290]: https://github.com/fluxcd/helm-operator/pull/290
+[#291]: https://github.com/fluxcd/helm-operator/pull/291
+[#292]: https://github.com/fluxcd/helm-operator/pull/292
+[#295]: https://github.com/fluxcd/helm-operator/pull/295
+[#296]: https://github.com/fluxcd/helm-operator/pull/296
+
 ## 1.0.0-rc8 (2020-01-25)
 
 > **Notice:** upgrading to this version from `<=0.10.x` by just
@@ -12,58 +95,58 @@
 
  - release: push returned error as condition on sync check
    failure
-   [fluxcd/helm-operator][#209]
+   [fluxcd/helm-operator#209][#209]
  - release: reject git source if URL and path are missing
-   [fluxcd/helm-operator][#223]
+   [fluxcd/helm-operator#223][#223]
  - helm: only hold repository config lock for duration of
    read so dry-runs are run in parallel again
-   [fluxcd/helm-operator][#225]
+   [fluxcd/helm-operator#225][#225]
  - release: use all set `rollback` values when performing
    a rollback operation
-   [fluxcd/helm-operator][#239]
+   [fluxcd/helm-operator#239][#239]
  - helm: do not include non-template files in chart data
    so that the generation of a `requirement.lock` due to
    a dependency update does not cause spurious upgrades
-   [fluxcd/helm-operator][#242]
+   [fluxcd/helm-operator#242][#242]
 
 ### Enhancements
 
  - release: allow `.spec.wait` to be set for upgrades
-   [fluxcd/helm-operator][#95]
+   [fluxcd/helm-operator#95][#95]
  - chartsync: support supplying Git HTTPS credentials
    using `secretRef`
-   [fluxcd/helm-operator][#172]
+   [fluxcd/helm-operator#172][#172]
  - status: retry status and condition updates on conflicts
-   [fluxcd/helm-operator][#210]
+   [fluxcd/helm-operator#210][#210]
  - release: allow `secretKeyRef` and `configMapKeyRef` to be
    selected from other namespaces using the `namespace` key
-   [fluxcd/helm-operator][#219]
+   [fluxcd/helm-operator#219][#219]
  - helm: only index missing repositories when fetching a
    chart from an URL
-   [fluxcd/helm-operator][#225]
+   [fluxcd/helm-operator#225][#225]
  - helm/v3: propagate main application logger to client
-   [fluxcd/helm-operator][#232]
+   [fluxcd/helm-operator#232][#232]
  - release: allow max history to be overridden using
    `.spec.maxHistory`
-   [fluxcd/helm-operator][#235]
+   [fluxcd/helm-operator#235][#235]
  - release: rely on Helm storage for determining when to
    upgrade after rolling back
-   [fluxcd/helm-operator][#239]
+   [fluxcd/helm-operator#239][#239]
 
 ### Maintenance and documentation
 
  - build: update Kubernetes Kind to `v0.7.0` and set
    Kubernetes `v1.14.10` for end-to-end tests
-   [fluxcd/helm-operator][#207]
+   [fluxcd/helm-operator#207][#207]
  - build: upgrade code-generator to Kubernetes 1.16.2
-   [fluxcd/helm-operator][#214]
+   [fluxcd/helm-operator#214][#214]
  - docs: update FAQ on Flux garbage collection
-   [fluxcd/helm-operator][#221]
+   [fluxcd/helm-operator#221][#221]
  - Pkg: update Flux to `v1.17.2-0.20200121140732-3903cf8e71c3`
-   [fluxcd/helm-operator][#230]
+   [fluxcd/helm-operator#230][#230]
  - Pkg: make `pkg/install` a Go module to reduce its
    dependencies
-   [fluxcd/helm-operator][#234]
+   [fluxcd/helm-operator#234][#234]
 
 
 ### Thanks

diff --git a/Makefile b/Makefile
@@ -22,6 +22,8 @@ endif
 CURRENT_OS_ARCH=$(shell echo `go env GOOS`-`go env GOARCH`)
 GOBIN?=$(shell echo `go env GOPATH`/bin)
 
+MAIN_GO_MODULE:=$(shell go list -m -f '{{ .Path }}')
+LOCAL_GO_MODULES:=$(shell go list -m -f '{{ .Path }}' all | grep $(MAIN_GO_MODULE))
 godeps=$(shell go list -deps -f '{{if not .Standard}}{{ $$dep := . }}{{range .GoFiles}}{{$$dep.Dir}}/{{.}} {{end}}{{end}}' $(1) | sed "s%${PWD}/%%g")
 
 HELM_OPERATOR_DEPS:=$(call godeps,./cmd/helm-operator/...)
@@ -43,7 +45,7 @@ realclean: clean
 	rm -rf ./cache
 
 test: test/bin/helm2 test/bin/helm3
-	PATH="${PWD}/bin:${PWD}/test/bin:${PATH}" go test ${TEST_FLAGS} $(shell go list ./... | grep -v "^github.com/weaveworks/flux/vendor" | sort -u)
+	PATH="${PWD}/bin:${PWD}/test/bin:${PATH}" go test ${TEST_FLAGS} $(shell go list $(patsubst %, %/..., $(LOCAL_GO_MODULES)) | sort -u)
 
 e2e: test/bin/helm2 test/bin/helm3 test/bin/kubectl test/e2e/bats build/.helm-operator.done
 	PATH="${PWD}/test/bin:${PATH}" CURRENT_OS_ARCH=$(CURRENT_OS_ARCH) test/e2e/run.bash
@@ -134,8 +136,14 @@ cache/bats-core-$(BATS_COMMIT).tar.gz:
 	# Use 2opremio's fork until https://github.com/bats-core/bats-core/pull/255 is merged
 	curl --fail -L -o $@ https://github.com/2opremio/bats-core/archive/$(BATS_COMMIT).tar.gz
 
+generate: generate-codegen generate-deploy
+
+generate-codegen:
+	./hack/update/generated.sh
+
 generate-deploy: pkg/install/generated_templates.gogen.go
 	cd deploy && go run ../pkg/install/generate.go deploy
+	cp ./deploy/flux-helm-release-crd.yaml ./chart/helm-operator/crds/helmrelease.yaml
 
 check-generated: generate-deploy pkg/install/generated_templates.gogen.go
 	git diff --exit-code -- pkg/install/generated_templates.gogen.go

diff --git a/README.md b/README.md
@@ -4,24 +4,42 @@
 [![GoDoc](https://godoc.org/github.com/fluxcd/helm-operator?status.svg)](https://godoc.org/github.com/fluxcd/helm-operator)
 [![Documentation](https://img.shields.io/badge/latest-documentation-informational)](https://docs.fluxcd.io/projects/helm-operator/en/latest/)
 
-The Helm Operator provides an extension to [Flux](https://github.com/fluxcd/flux)
-that automates Helm Chart releases in a GitOps manner.
-A Chart release is described through a Kubernetes custom resource named HelmRelease. 
-The Flux daemon synchronizes these resources from git to the cluster, 
-and the Helm Operator makes sure Helm charts are released as specified in the resources.
+The Helm operator is a [Kubernetes operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/),
+allowing one to declaratively manage Helm chart releases. Combined with
+[Flux](https://github.com/fluxcd/flux) this can be utilized to automate
+releases in a GitOps manner, but the usage of Flux is not a strict
+requirement.
+
+The desired state of a Helm release is described through a Kubernetes
+Custom Resource named `HelmRelease`. Based on the creation, mutation or
+removal of a `HelmRelease` resource in the cluster, Helm actions are
+performed by the operator.
 
 ![GitOps Helm Operator](docs/_files/fluxcd-helm-operator-diagram.png)
 
 ## Helm Operator features
 
-* declarative helm install/upgrade/delete of charts
-* pulls charts from public or private Helm repositories over HTTPS
-* pulls charts from public or private Git repositories over SSH
-* chart release values can be specified inline in the HelmRelease object or via secrets, configmaps or URLs
-* automated chart upgrades based on container image tag policies (requires Flux)
-* automatic purging on chart install failures
-* automatic rollback on chart upgrade failures
-* supports both Helm v2 and v3
+* Declarative install, upgrade, and delete of Helm releases
+* Pulls chart from _any_ chart source;
+  * Public or private Helm repositories over HTTP/S
+  * Public or private Git repositories over HTTPS or SSH
+  * Any other public or private chart source using one of the available
+    [Helm downloader plugins](https://helm.sh/docs/topics/plugins/#downloader-plugins)
+* Allows Helm values to be specified;
+  * In-line in the `HelmRelease` resource
+  * In (external) sources, e.g. `ConfigMap` and `Secret` resources,
+    or a (local) URL
+* Automated purging on release install failures
+* Automated (optional) rollback on upgrade failures
+* Automated image upgrades [using Flux](https://docs.fluxcd.io/en/latest/references/helm-operator-integration.html)
+* Automated (configurable) chart dependency updates for Helm charts
+  from Git sources on install or upgrade
+* Detection and recovery from Helm storage mutations
+  (e.g. a manual Helm release that was made but conflicts with the
+  declared configuration for the release)
+* Parallel and scalable processing of different `HelmRelease` resources
+  using workers
+* Supports both Helm 2 and 3
 
 ## Get started with the Helm Operator
 

diff --git a/chart/helm-operator/CHANGELOG.md b/chart/helm-operator/CHANGELOG.md
@@ -1,3 +1,12 @@
+## 0.6.0 (2020-01-26)
+
+### Improvements
+
+ - Update Helm Operator to `1.0.0-rc8`
+   [fluxcd/helm-operator#244](https://github.com/fluxcd/helm-operator/pull/244)
+ - Allow pod annotations, labels and account annotations to be set
+   [fluxcd/helm-operator#229](https://github.com/fluxcd/helm-operator/pull/229)
+
 ## 0.5.0 (2020-01-10)
 
 ### Improvements

diff --git a/chart/helm-operator/Chart.yaml b/chart/helm-operator/Chart.yaml
@@ -1,6 +1,6 @@
 apiVersion: v1
-appVersion: "1.0.0-rc8"
-version: 0.5.0
+appVersion: "1.0.0-rc9"
+version: 0.6.0
 kubeVersion: ">=1.11.0-0"
 name: helm-operator
 description: Flux Helm Operator is a CRD controller for declarative helming

diff --git a/chart/helm-operator/README.md b/chart/helm-operator/README.md
@@ -164,6 +164,47 @@ helm upgrade -i helm-operator fluxcd/helm-operator \
 
 The deploy key naming convention is `<Flux Release Name>-git-deploy`.
 
+## Use Helm downloader plugins
+
+Helm downloader plugins like [`hypnoglow/helm-s3`](https://github.com/hypnoglow/helm-s3)
+and [`hayorov/helm-gcs`](https://github.com/hayorov/helm-gcs) make it possible
+to extend the protocols Helm recognizes to e.g. pull charts from a S3 bucket.
+
+The chart offers an utility to install plugins before starting the operator
+using init containers:
+
+```sh
+helm upgrade -i helm-operator fluxcd/helm-operator \
+--namespace fluxcd \
+--set initPlugins.enable=true \
+--set 'initPlugins.plugins[0].plugin=https://github.com/hypnoglow/helm-s3.git' \
+--set 'initPlugins.plugins[0].version=0.9.2' \
+--set 'initPlugins.plugins[0].helmVersion=v3' \
+```
+
+> **Note:** most plugins assume credentials are available on the system they run on,
+> make sure those are available at the expected paths using e.g. `extraVolumes` and
+> `extraVolumeMounts`.
+
+You should now be able to make use of the protocol added by the plugin:
+
+```sh
+cat <<EOF | kubectl apply -f -
+apiVersion: helm.fluxcd.io/v1
+kind: HelmRelease
+metadata:
+  name: chart-from-s3
+  namespace: default
+spec:
+  chart:
+    repository: s3://bucket-name/charts
+    name: chart
+    version: 0.1.0
+  values:
+    replicaCount: 1
+EOF
+```
+
 ## Uninstall
 
 To uninstall/delete the `helm-operator` deployment:
@@ -238,10 +279,13 @@ The following tables lists the configurable parameters of the Flux chart and the
 | `configureRepositories.secretName`                | `flux-helm-repositories`                             | Name of the secret containing the contents of the `repositories.yaml` file
 | `configureRepositories.cacheName`                 | `repositories-cache`                                 | Name for the repository cache volume
 | `configureRepositories.repositories`              | `None`                                               | List of custom Helm repositories to add. If non empty, the corresponding secret with a `repositories.yaml` will be created
+| `initPlugins.enable`                              | `false`                                              | Enable the initialization of Helm plugins using init containers
+| `initPlugins.cacheVolumeName`                     | `plugins-cache`                                      | Name for the plugins cache volume
+| `initPlugins.plugins`                             | `None`                                               | List of Helm plugins to initialize before starting the operator. If non empty, an init container will be added for every entry.
 | `kube.config`                                     | `None`                                               | Override for kubectl default config in the Helm Operator pod(s).
 | `prometheus.enabled`                              | `false`                                              | If enabled, adds prometheus annotations to Helm Operator pod(s)
 | `prometheus.serviceMonitor.create`                | `false`                                              | Set to true if using the Prometheus Operator
 | `prometheus.serviceMonitor.interval`              | ``                                                   | Interval at which metrics should be scraped
 | `prometheus.serviceMonitor.namespace`             | ``                                                   | The namespace where the ServiceMonitor is deployed
 | `prometheus.serviceMonitor.additionalLabels`      | `{}`                                                 | Additional labels to add to the ServiceMonitor
-
+| `initContainers`                                  | `[]`                                                 | Init containers and their specs