bump/add new revisions for opsportal/kommander charts

[juliangieseke]

bumping kommander & ops portal charts by adding new revisions.

I made two commits to keep changes visible, first commit is duplicating files, second commit does the change: dd5e4e7

@shaneutt last double checking that this is correct, mind having a look? 🙇

[sebbrandt87]

/lgtm
checked against mesosphere/charts
https://github.com/mesosphere/charts/blob/master/stable/opsportal/Chart.yaml#L6
and
https://github.com/mesosphere/charts/blob/master/stable/kommander/Chart.yaml#L10

[juliangieseke]

hm, not sure what the CI issue is about:

[15:39:22][Step 1/1] configmap/local-path-config created
[15:47:59][Step 1/1] panic: test timed out after 10m0s
[15:47:59][Step 1/1] 
[15:47:59][Step 1/1] goroutine 4069 [running]:
[15:47:59][Step 1/1] testing.(*M).startAlarm.func1()
[15:47:59][Step 1/1] 	/usr/local/go/src/testing/testing.go:1377 +0x11c
[15:47:59][Step 1/1] created by time.goFunc
[15:47:59][Step 1/1] 	/usr/local/go/src/time/sleep.go:168 +0x52

Does this mean the test "configmap/local-path-config created" has timed out? 🤔

[shaneutt]

hm, not sure what the CI issue is about:

[15:39:22][Step 1/1] configmap/local-path-config created
[15:47:59][Step 1/1] panic: test timed out after 10m0s
[15:47:59][Step 1/1] 
[15:47:59][Step 1/1] goroutine 4069 [running]:
[15:47:59][Step 1/1] testing.(*M).startAlarm.func1()
[15:47:59][Step 1/1] 	/usr/local/go/src/testing/testing.go:1377 +0x11c
[15:47:59][Step 1/1] created by time.goFunc
[15:47:59][Step 1/1] 	/usr/local/go/src/time/sleep.go:168 +0x52

Does this mean the test "configmap/local-path-config created" has timed out? thinking

The error output in this (very young) CI code is very lacking right now. This error means that the test failed because the addons could not fully provision.

We intend to follow up on this and improve the experience as per DCOS-61626, but until then its customary to run the tests locally and monitor the addons to determine the source of the issue.

[sebbrandt87]

running tests locally

kommander            karma-configmap-cleanup-msjc2                                                0/1     Error     2          28s
kommander            kcl-cm-86788c5db9-pwp6f                                                      2/2     Running   0          2m43s
kommander            kcl-tfcb-6b98cd5997-bgndb                                                    1/1     Running   0          2m43s
kommander            kcl-utility-apiserver-5bb48f4f8d-kv5tp                                       1/1     Running   0          2m43s
kommander            kcl-webhook-569c99bc6c-jgc6m                                                 1/1     Running   0          2m43s
kommander            kommander-kubeaddons-grafana-6656bdb6c6-l82ck                                2/2     Running   0          2m43s
kommander            kommander-kubeaddons-karma-68fcb69494-ql6dz                                  1/1     Running   0          2m43s
kommander            kommander-kubeaddons-kommander-ui-5ddd874f5d-kcd99                           1/1     Running   0          2m43s
kommander            kommander-kubeaddons-kubeaddons-catalog-7df9885bff-7cv6s                     1/1     Running   0          2m43s
kommander            kommander-kubeaddons-thanos-query-5bd88795c5-8z2bz                           1/1     Running   0          2m43s
kommander            kubefed-admission-webhook-7fdcc5b56f-2655s                                   1/1     Running   0          2m43s
kommander            kubefed-controller-manager-75bdd8c787-rskfx                                  1/1     Running   0          2m43s
kommander            kubefed-controller-manager-75bdd8c787-swst4                                  1/1     Running   0          2m43s
kube-system          coredns-5c98db65d4-r94x4                                                     1/1     Running   0          9m10s
kube-system          coredns-5c98db65d4-zk5zb                                                     1/1     Running   0          9m10s
kube-system          etcd-b1041817-593c-4a61-8e96-58610a538cec-control-plane                      1/1     Running   0          8m26s
kube-system          kindnet-lt95w                                                                1/1     Running   0          9m10s
kube-system          kube-apiserver-b1041817-593c-4a61-8e96-58610a538cec-control-plane            1/1     Running   0          8m22s
kube-system          kube-controller-manager-b1041817-593c-4a61-8e96-58610a538cec-control-plane   1/1     Running   0          8m18s
kube-system          kube-proxy-vxgtm                                                             1/1     Running   0          9m10s
kube-system          kube-scheduler-b1041817-593c-4a61-8e96-58610a538cec-control-plane            1/1     Running   0          8m32s
kube-system          kubeaddons-controller-manager-5d9957d9fb-hx5n6                               1/1     Running   0          8m28s
kube-system          tiller-deploy-54f7455d59-r92sw                                               1/1     Running   0          7m59s
local-path-storage   local-path-provisioner-84f4c8b584-lkf8l                                      1/1     Running   0          8m26s

kc logs -f -n kommander karma-configmap-cleanup-msjc2
Error from server (Forbidden): configmaps "kommander-kubeaddons-config" is forbidden: User "system:serviceaccount:kommander:kommander-kubeaddons" cannot delete resource "configmaps" in API group "" in the namespace "kommander"

so correct me if I am wrong, this is not related to kommander itself at all, but part of the testing process. right?

[shaneutt]

so correct me if I am wrong, this is not related to kommander itself at all, but part of the testing process. right?

Looks like a problem in the underlying chart which we've just now caught.

Since this didn't occur in previous CI runs it would seem that a recent chart update introduced this problem?

That being the case then CI would appear to have done it's job and caught a problem and we should fix the chart and update this PR with a chart version that deploys and tears down fully.

Let me know if you feel there's any context I've missed here?

[sebbrandt87]

didn't checked this karma thing ... will do now

[shaneutt]

When CI is happy, I'm happy 👍

[juliangieseke]

rebased, seems like opsportal got merged in a separate PR, CI for kommander still being red. I cant make a lot of sense of that CI issue :(

[shaneutt]

rebased, seems like opsportal got merged in a separate PR, CI for kommander still being red. I cant make a lot of sense of that CI issue :(

Ok, np I'll take a look at the CI issue today @juliangieseke and get it sorted out 👍

[shaneutt]

Thanks for the heads up, I'm going to put this back in wip for now while the chart is investigated, let me know how I can be helpful!

[gracedo]

@shaneutt @juliangieseke Let's see if getting mesosphere/charts#386 merged and bumping to that chart version here fixes it? I'm not sure why the cleanup job is failing now and not before, but there are some rbac improvements in that PR that could fix this.

[sebbrandt87]

The problem appears to be occuring in cleanup @juliangieseke.

The karma-configmap-cleanup pods are going into crash loop on cleanup logging the following (from the pod):

Error from server (Forbidden): configmaps "kommander-kubeaddons-config" is forbidden: User "system:serviceaccount:kommander:kommander-kubeaddons" cannot delete resource "configmaps" in API group "" in the namespace "kommander"

It would appear that we still have some permissions issue in the chart that we need to correct.

it was the same error for me #100 (comment)

[juliangieseke]

@sebbrandt87 ah ok, I've been under the impression that mesosphere/charts#384 should have fixed the issue…
@gracedo is mesosphere/charts#386 fixing the issue?

[gracedo]

mesosphere/charts#384 are changes to kommander-karma and kommander-thanos charts, which are dependencies in the kommander chart - those deps are actually bumped in mesosphere/charts#386 so the changes wouldn't be in until that PR is merged and we bump the version of kommander here

[gracedo]

@juliangieseke just merged mesosphere/charts#386 - let's update the kommander chart version to 0.3.21 and see if it fixes things

[juliangieseke]

thanks for heads up @gracedo

lets wait for CI 🤞

[juliangieseke]

CI run to have a look at: https://teamcity.mesosphere.io/viewLog.html?buildId=2566529&buildTypeId=kubeaddons_KonvoyAddons

(Ive bumped ui again to include a fix)

[juliangieseke]

Its green 🎉 cc @shaneutt @gracedo