--policies=* does not appear to be working

[stefansedich]

Given the following deployment:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: opa
  namespace: opa
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: opa
    spec:
      containers:
      - name: opa
        image: openpolicyagent/opa
        args:
        - "run"
        - "--server"
        ports:
        - name: http
          containerPort: 8181
      - name: kube-mgmt
        image: openpolicyagent/kube-mgmt:0.10
        args:
          - --enable-policies=true
          - --policies=*
          - --require-policy-label=true

And given the following configmap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: test
  namespace: test
  labels:
    openpolicyagent.org/policy: rego
data:
  test.rego: |
    package kubernetes

    example = "Hello, Kubernetes!"

kube-mgmt does not appear to be loading the configmap at all, if I were to set --policies=test with an explicit namespace it will load the configmaps fine.

I was under the impression that --policies=* will load from any namespace?

[ashutosh-narkar]

Yes with --policies=* kube-mgmt should look for policies in all namespaces. Since you also specified --require-policy-label=true, have you also labelled the configmaps in the other namespaces ?

[stefansedich]

Yup I had done that @ashutosh-narkar, I could only get it to pick up poliies when using --policies=a,b,c and being explicit with namespaces.

[stefansedich]

@ashutosh-narkar could I be missing anything else here? have tried various things and had no luck with --policies=* it is working only if I use --policies with an explicit namespace.

[ashutosh-narkar]

You can remove --enable-policies=true as it's true by default (https://github.com/open-policy-agent/kube-mgmt/blob/master/cmd/kube-mgmt/main.go#L71). Also may be try enclosing the arguments in quotes eg "--policies=*". Let me know if this doesn't work and I can try this out.

[stefansedich]

No luck @ashutosh-narkar, no combination of quotes gets it working using --policies=* but as soon as I change it to --policies=test being the namespace my policy is in it loads it fine.

[ashutosh-narkar]

Ok @stefansedich , I will try this out too.

[patoarvizu]

I'm seeing similar behavior but it seems to be inconsistent. One thing I noticed that I could kind of consistently reproduce but didn't make sense is that if I launch with --policies=* --require-policy-label=false, it won't discover policies even if they're in the opa namespace and have openpolicyagent.org/policy: rego on the ConfigMap. However, if I then set --require-policy-label=false, the ConfigMaps did get the openpolicyagent.org/policy-status: '{"status":"ok"}' annotation. It doesn't make sense that switch from enforcing the label no not enforcing it will suddenly make it work.

I couldn't reproduce this behavior consistently, but I seemed to be able to reproduce it very often. I was applying changes and checking the objects in quick succession, so it's possible there some race condition that made the behavior inconsistent.

I tested on both 0.10 and 0.11, on Kubernetes (k3s) 1.15. I can try running more tests later if I have time.

[ashutosh-narkar]

Thanks @patoarvizu for helping to reproduce the issue. Sorry about the delay in looking into this. I'll try to reproduce this myself tomorrow.

[ashutosh-narkar]

I could reproduce the issue and the implementation for the --policies=* feature doesn't seem to be doing what it's supposed to. Working on fix for it.

[ashutosh-narkar]

PR: #64.

As mentioned in the PR commit, the --policies=* option was breaking the config map matching logic. It would appear to work (mistakenly) in cases where --policies=* is set and --require-policy-label=false but the config map had the label openpolicyagent.org/policy=rego. This behavior was observed by @patoarvizu as well.

Also it would be recommended to use the --require-policy-label=true option in conjunction with --policies=*. You would also need to give OPA/kube-mgmt a ClusterRole that allows it to annotate config maps in all namespaces.