Merge pull request #1734 from 2opremio/release/1.10.1

Release Flux v1.10.1

CHANGELOG-helmop.md

@@ -1,10 +1,17 @@
-## 0.5.4 (TBA)
+## 0.6.0 (2019-02-07)
 
 ### Improvements
 
  - Add option to limit the Helm operator to a single namespace
    [weaveworks/flux#1664](https://github.com/weaveworks/flux/pull/1664)
 
+### Thanks
+
+Without the contributions of @brandon-bethke-neudesic, @errordeveloper,
+@ncabatoff, @stefanprodan, @squaremo, and feedback of our
+[#flux](https://slack.weave.works/) inhabitants this release would not
+have been possible -- thanks to all of you!
+
 ## 0.5.3 (2019-01-14)
 
 ### Improvements

CHANGELOG.md

@@ -1,6 +1,32 @@
 This is the changelog for the Flux daemon; the changelog for the Helm
 operator is in [./CHANGELOG-helmop.md](./CHANGELOG-helmop.md).
 
+## 1.10.1 (2019-02-13)
+
+This release provides a deeper integration with Azure (DevOps Git hosts 
+and ACR) and allows configuring how `fluxctl` finds `fluxd` (useful for 
+clusters with multiple fluxd installations).
+
+### Improvements
+
+- Support Azure DevOps Git hosts
+  [weaveworks/flux#1729][#1729]
+  [weaveworks/flux#1731][#1731]
+- Use AKS credentials for ACR
+  [weaveworks/flux#1694][#1694]
+- Make port forward label selector configurable
+  [weaveworks/flux#1727][#1727]
+
+### Thanks
+
+Lots of thanks to @alanjcastonguay, @hiddeco, and @sarath-p for their 
+contributions to this release.
+
+[#1694]: https://github.com/weaveworks/flux/pull/1694
+[#1727]: https://github.com/weaveworks/flux/pull/1727
+[#1729]: https://github.com/weaveworks/flux/pull/1729
+[#1731]: https://github.com/weaveworks/flux/pull/1731
+
 ## 1.10.0 (2019-02-07)
 
 This release adds the `--registry-exclude-image` flag for excluding

chart/flux/CHANGELOG.md

@@ -1,10 +1,33 @@
-## 0.6.1 (TBA)
+## 0.6.2 (2019-02-11)
 
 ### Improvements
 
+ - Allow chart images to be pulled from a private container registry
+   [weaveworks/flux#1718](https://github.com/weaveworks/flux/pull/1718)
+
+### Bug fixes
+
+ - Fix helm-op allow namespace flag mapping
+   [weaveworks/flux#1724](https://github.com/weaveworks/flux/pull/1724)
+
+## 0.6.1 (2019-02-07)
+
+### Improvements
+
+ - Updated Flux to `1.10.0` and Helm operator to `0.6.0`
+   [weaveworks/flux#1713](https://github.com/weaveworks/flux/pull/1713)
+ - Add option to exclude container images
+   [weaveworks/flux#1659](https://github.com/weaveworks/flux/pull/1659)
+ - Add option to mount custom `repositories.yaml`
+   [weaveworks/flux#1671](https://github.com/weaveworks/flux/pull/1671)
  - Add option to limit the Helm operator to a single namespace
    [weaveworks/flux#1664](https://github.com/weaveworks/flux/pull/1664)
 
+### Bug fixes
+
+ - Fix custom SSH secret mapping
+   [weaveworks/flux#1710](https://github.com/weaveworks/flux/pull/1710)
+
 ## 0.6.0 (2019-01-14)
 
 **Note** To fix the connectivity problems between Flux and memcached we've changed the 

chart/flux/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
-appVersion: "1.9.0"
-version: 0.6.0
+appVersion: "1.10.0"
+version: 0.6.2
 kubeVersion: ">=1.9.0-0"
 name: flux
 description: Flux is a tool that automatically ensures that the state of a cluster matches what is specified in version control

chart/flux/README.md

@@ -187,6 +187,7 @@ The following tables lists the configurable parameters of the Weave Flux chart a
 | `image.tag` | Image tag | `<VERSION>`
 | `replicaCount` | Number of flux pods to deploy, more than one is not desirable. | `1`
 | `image.pullPolicy` | Image pull policy | `IfNotPresent`
+| `image.pullSecret` | Image pull secret | `None`
 | `resources.requests.cpu` | CPU resource requests for the flux deployment | `50m`
 | `resources.requests.memory` | Memory resource requests for the flux deployment | `64Mi`
 | `resources.limits` | CPU/memory resource limits for the flux deployment | `None`
@@ -222,16 +223,21 @@ The following tables lists the configurable parameters of the Weave Flux chart a
 | `registry.ecr.region` | Restrict ECR scanning to these AWS regions; if empty, only the cluster's region will be scanned | `None`
 | `registry.ecr.includeId` | Restrict ECR scanning to these AWS account IDs; if empty, all account IDs that aren't excluded may be scanned | `None`
 | `registry.ecr.excludeId` | Do not scan ECR for images in these AWS account IDs; the default is to exclude the EKS system account | `602401143452`
+| `registry.acr.enabled` | Mount `azure.json` via HostPath into the Flux Pod, enabling Flux to use AKS's service principal for ACR authentication | `false`
+| `registry.acr.hostPath` | Alternative location of `azure.json` on the host | `/etc/kubernetes/azure.json`
 | `memcached.verbose` | Enable request logging in memcached | `false`
 | `memcached.maxItemSize` | Maximum size for one item | `1m`
 | `memcached.maxMemory` | Maximum memory to use, in megabytes | `64`
+| `memcached.pullSecret` | Image pull secret | `None`
+| `memcached.repository` | Image repository | `memcached`
 | `memcached.resources` | CPU/memory resource requests/limits for memcached | `None`
 | `helmOperator.create` | If `true`, install the Helm operator | `false`
 | `helmOperator.createCRD` | Create the `v1beta1` and `v1alpha2` flux CRDs. Dependent on `helmOperator.create=true` | `true`
 | `helmOperator.repository` | Helm operator image repository | `quay.io/weaveworks/helm-operator`
 | `helmOperator.tag` | Helm operator image tag | `<VERSION>`
 | `helmOperator.replicaCount` | Number of helm operator pods to deploy, more than one is not desirable. | `1`
 | `helmOperator.pullPolicy` | Helm operator image pull policy | `IfNotPresent`
+| `helmOperator.pullSecret` | Image pull secret | `None`
 | `helmOperator.updateChartDeps` | Update dependencies for charts | `true`
 | `helmOperator.git.pollInterval` | Period at which to poll git repo for new commits | `git.pollInterval`
 | `helmOperator.git.timeout` | Duration after which git operations time out | `git.timeout`

chart/flux/templates/deployment.yaml

@@ -24,6 +24,10 @@ spec:
         release: {{ .Release.Name }}
     spec:
       serviceAccountName: {{ template "flux.serviceAccountName" . }}
+      {{- if .Values.image.pullSecret }}
+      imagePullSecrets:
+        - name: {{ .Values.image.pullSecret }}
+      {{- end }}
       volumes:
       - name: kubedir
         configMap:
@@ -45,6 +49,12 @@ spec:
       - name: git-keygen
         emptyDir:
           medium: Memory
+      {{- if .Values.registry.acr.enabled }}
+      - name: acr-credentials
+        hostPath:
+          path: "{{ .Values.registry.acr.hostPath }}"
+          type: ""
+      {{- end }}
       containers:
         - name: {{ .Chart.Name }}
           image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
@@ -66,6 +76,11 @@ spec:
             readOnly: true
           - name: git-keygen
             mountPath: /var/fluxd/keygen
+          {{- if .Values.registry.acr.enabled }}
+          - name: acr-credentials
+            mountPath: /etc/kubernetes/azure.json
+            readOnly: true
+          {{- end }}
           env:
           - name: KUBECONFIG
             value: /root/.kubectl/config
@@ -74,7 +89,7 @@ spec:
           {{- end }}
           args:
           - --ssh-keygen-dir=/var/fluxd/keygen
-          - --k8s-secret-name={{ template "flux.fullname" . }}-git-deploy
+          - --k8s-secret-name={{ .Values.git.secretName | default (printf "%s-git-deploy" (include "flux.fullname" .)) }}
           - --memcached-hostname={{ template "flux.fullname" . }}-memcached
           {{- if .Values.memcached.createClusterIP }}
           - --memcached-service=

chart/flux/templates/helm-operator-deployment.yaml

@@ -26,6 +26,10 @@ spec:
         release: {{ .Release.Name }}
     spec:
       serviceAccountName: {{ template "flux.serviceAccountName" . }}
+      {{- if .Values.helmOperator.pullSecret }}
+      imagePullSecrets:
+        - name: {{ .Values.helmOperator.pullSecret }}
+      {{- end }}
       volumes:
       {{- if .Values.ssh.known_hosts }}
       - name: sshdir
@@ -100,7 +104,7 @@ spec:
         - --charts-sync-interval={{ .Values.helmOperator.chartsSyncInterval }}
         - --update-chart-deps={{ .Values.helmOperator.updateChartDeps }}
         - --log-release-diffs={{ .Values.helmOperator.logReleaseDiffs }}
-        {{- if .Values.helmOperator.namespace }}
+        {{- if .Values.helmOperator.allowNamespace }}
         - --allow-namespace={{ .Values.helmOperator.allowNamespace }}
         {{- end }}
         - --tiller-namespace={{ .Values.helmOperator.tillerNamespace }}

chart/flux/templates/memcached.yaml

@@ -21,6 +21,10 @@ spec:
         app: {{ template "flux.name" . }}-memcached
         release: {{ .Release.Name }}
     spec:
+      {{- if .Values.memcached.pullSecret }}
+      imagePullSecrets:
+        - name: {{ .Values.memcached.pullSecret }}
+      {{- end }}
       containers:
       - name: memcached
         image: {{ .Values.memcached.repository }}:{{ .Values.memcached.tag }}

chart/flux/values.yaml

@@ -7,8 +7,9 @@ replicaCount: 1
 
 image:
   repository: quay.io/weaveworks/flux
-  tag: 1.9.0
+  tag: 1.10.0
   pullPolicy: IfNotPresent
+  pullSecret:
 
 service:
   type: ClusterIP
@@ -19,8 +20,9 @@ helmOperator:
   create: false
   createCRD: true
   repository: quay.io/weaveworks/helm-operator
-  tag: 0.5.3
+  tag: 0.6.0
   pullPolicy: IfNotPresent
+  pullSecret:
   # Limit the operator scope to a single namespace
   allowNamespace:
   # Update dependencies for charts
@@ -52,7 +54,7 @@ helmOperator:
     # generate a SSH key named identity: ssh-keygen -q -N "" -f ./identity
     # create a Kubernetes secret: kubectl -n flux create secret generic helm-ssh --from-file=./identity
     # delete the private key: rm ./identity
-    # add ./identity.pub as a read-only deployment key in your GitHub repo where the charts are
+    # add ./identity.pub as a read-only deployment key in your Git repo where the charts are
     # set the secret name (helm-ssh) below
     secretName: ""
   # Additional environment variables to set
@@ -114,7 +116,7 @@ git:
   # generate a SSH key named identity: ssh-keygen -q -N "" -f ./identity
   # create a Kubernetes secret: kubectl -n flux create secret generic flux-ssh --from-file=./identity
   # delete the private key: rm ./identity
-  # add ./identity.pub as a read-only deployment key in your GitHub repo where the charts are
+  # add ./identity.pub as a deployment key with write access in your Git repo
   # set the secret name (flux-ssh) below
   secretName: ""
 
@@ -138,10 +140,15 @@ registry:
     region:
     includeId:
     excludeId:
+  # Azure ACR settings
+  acr:
+    enabled: false
+    hostPath: /etc/kubernetes/azure.json
 
 memcached:
   repository: memcached
   tag: 1.4.25
+  pullSecret:
   createClusterIP: true
   verbose: false
   maxItemSize: 1m

cmd/fluxctl/root_cmd.go

@@ -21,6 +21,7 @@ type rootOpts struct {
 	URL       string
 	Token     string
 	Namespace string
+	Labels    map[string]string
 	API       api.Server
 }
 
@@ -51,6 +52,7 @@ Workflow:
 const (
 	envVariableURL        = "FLUX_URL"
 	envVariableNamespace  = "FLUX_FORWARD_NAMESPACE"
+	envVariableLabels     = "FLUX_FORWARD_LABELS"
 	envVariableToken      = "FLUX_SERVICE_TOKEN"
 	envVariableCloudToken = "WEAVE_CLOUD_TOKEN"
 	defaultURLGivenToken  = "https://cloud.weave.works/api/flux"
@@ -67,6 +69,8 @@ func (opts *rootOpts) Command() *cobra.Command {
 
 	cmd.PersistentFlags().StringVar(&opts.Namespace, "k8s-fwd-ns", "default",
 		fmt.Sprintf("Namespace in which fluxd is running, for creating a port forward to access the API. No port forward will be created if a URL or token is given. You can also set the environment variable %s", envVariableNamespace))
+	cmd.PersistentFlags().StringToStringVar(&opts.Labels, "k8s-fwd-labels", map[string]string{"app": "flux"},
+		fmt.Sprintf("Labels used to select the fluxd pod a port forward should be created for. You can also set the environment variable %s", envVariableLabels))
 	cmd.PersistentFlags().StringVarP(&opts.URL, "url", "u", "",
 		fmt.Sprintf("Base URL of the flux API (defaults to %q if a token is provided); you can also set the environment variable %s", defaultURLGivenToken, envVariableURL))
 	cmd.PersistentFlags().StringVarP(&opts.Token, "token", "t", "",
@@ -98,27 +102,26 @@ func (opts *rootOpts) PersistentPreRunE(cmd *cobra.Command, _ []string) error {
 		return nil
 	}
 
-	opts.Namespace = getFromEnvIfNotSet(cmd.Flags(), "k8s-fwd-ns", opts.Namespace, envVariableNamespace)
-	opts.Token = getFromEnvIfNotSet(cmd.Flags(), "token", opts.Token, envVariableToken, envVariableCloudToken)
-	opts.URL = getFromEnvIfNotSet(cmd.Flags(), "url", opts.URL, envVariableURL)
+	setFromEnvIfNotSet(cmd.Flags(), "k8s-fwd-ns", envVariableNamespace)
+	setFromEnvIfNotSet(cmd.Flags(), "k8s-fwd-labels", envVariableLabels)
+	setFromEnvIfNotSet(cmd.Flags(), "token", envVariableToken, envVariableCloudToken)
+	setFromEnvIfNotSet(cmd.Flags(), "url", envVariableURL)
 
 	if opts.Token != "" && opts.URL == "" {
 		opts.URL = defaultURLGivenToken
 	}
 
 	if opts.URL == "" {
 		portforwarder, err := tryPortforwards(opts.Namespace, metav1.LabelSelector{
+			MatchLabels: opts.Labels,
+		}, metav1.LabelSelector{
 			MatchExpressions: []metav1.LabelSelectorRequirement{
 				metav1.LabelSelectorRequirement{
 					Key:      "name",
 					Operator: metav1.LabelSelectorOpIn,
 					Values:   []string{"flux", "fluxd", "weave-flux-agent"},
 				},
 			},
-		}, metav1.LabelSelector{
-			MatchLabels: map[string]string{
-				"app": "flux",
-			},
 		})
 		if err != nil {
 			return err
@@ -135,14 +138,13 @@ func (opts *rootOpts) PersistentPreRunE(cmd *cobra.Command, _ []string) error {
 	return nil
 }
 
-func getFromEnvIfNotSet(flags *pflag.FlagSet, flagName, value string, envNames ...string) string {
+func setFromEnvIfNotSet(flags *pflag.FlagSet, flagName string, envNames ...string) {
 	if flags.Changed(flagName) {
-		return value
+		return
 	}
 	for _, envName := range envNames {
 		if env := os.Getenv(envName); env != "" {
-			return env
+			flags.Set(flagName, env)
 		}
 	}
-	return value // not changed, so presumably the default
 }

deploy-helm/flux-helm-release-crd.yaml

@@ -31,6 +31,8 @@ spec:
               format: int64
             resetValues:
               type: boolean
+            skipDepUpdate:
+              type: boolean
             valueFileSecrets:
               type: array
               properties:

deploy-helm/helm-operator-deployment.yaml

@@ -62,7 +62,7 @@ spec:
         # There are no ":latest" images for helm-operator. Find the most recent
         # release or image version at https://quay.io/weaveworks/helm-operator
         # and replace the tag here.
-        image: quay.io/weaveworks/helm-operator:0.5.3
+        image: quay.io/weaveworks/helm-operator:0.6.0
         imagePullPolicy: IfNotPresent
         ports:
         - name: http

deploy-helm/weave-cloud-helm-operator-deployment.yaml

@@ -27,7 +27,7 @@ spec:
             secretName: flux-git-deploy
       containers:
         - name: flux-helm-operator
-          image: quay.io/weaveworks/helm-operator:0.5.3
+          image: quay.io/weaveworks/helm-operator:0.6.0
           imagePullPolicy: IfNotPresent
           args:
             - --git-timeout=20s

deploy/flux-deployment.yaml

@@ -54,7 +54,7 @@ spec:
         # There are no ":latest" images for flux. Find the most recent
         # release or image version at https://quay.io/weaveworks/flux
         # and replace the tag here.
-        image: quay.io/weaveworks/flux:1.10.0
+        image: quay.io/weaveworks/flux:1.10.1
         imagePullPolicy: IfNotPresent
         resources:
           requests:

docker/Dockerfile.flux

@@ -6,7 +6,7 @@ RUN apk add --no-cache openssh ca-certificates tini 'git>=2.3.0'
 
 # Add git hosts to known hosts file so we can use
 # StrickHostKeyChecking with git+ssh
-RUN ssh-keyscan github.com gitlab.com bitbucket.org >> /etc/ssh/ssh_known_hosts
+RUN ssh-keyscan github.com gitlab.com bitbucket.org ssh.dev.azure.com >> /etc/ssh/ssh_known_hosts
 
 # Verify newly added known_hosts (man-in-middle mitigation)
 ADD ./verify_known_hosts.sh /home/flux/verify_known_hosts.sh