Are you aware of ibotty/openshift-letsencrypt

[dustymabe]

I opened a similar issue on that repo as well: ibotty/openshift-letsencrypt#23

[ibotty]

I am the primary author of the other effort.

I don't know how joining efforts might work: the implementation is utterly different. I guess we could unify the way to mark a route as needing a certificate. I would also be fine with blessing one or the other. We should write down a detailed comparison.

[andrewklau]

If I recall correctly, the main noticeable differences are this repository runs more at an operator level. It requires a cluster level permission but it will automatically create your acme-account, certs and work cross namespace. Each namespace gets it's own acme-account created automatically.

Where as ibotty/openshift-letsencrypt will only work on an individual namespace.

[tnozicka]

openshift-acme supports both cluster level and individual namespaces (https://github.com/tnozicka/openshift-acme/blob/master/pkg/cmd/cmd.go#L94)

as @ibotty noted the implementations differ completely. They seem to be using the same technique of special routes, instead of requiring you to deploy a modified router, which is good.

Our annotation is named to be compatible with (https://github.com/jetstack/kube-lego#how-kube-lego-works). Btw. they went with the approach of requiring you to deploy a modified router.

[dustymabe]

hey @tnozicka and @ibotty - could we come up with a list of use cases for both https://github.com/ibotty/openshift-letsencrypt and https://github.com/tnozicka/openshift-acme and make sure that openshift-acme handles all use cases effectively?

@ibotty if this repo handles all use cases do you think it would be worth contributing to this implementation in the future?

[ibotty]

Maybe. I don't particularly enjoy programming go though. What privileges does openshift-acme need? I would not be comfortable requiring way more. Some of my clients are pretty paranoid (rightly so!).

[tnozicka]

It requires just the privileges needed to create/modify Routes and Secrets. If you are setting it in your namespace only, no elevated privileges are needed.

If you are setting it up for the whole cluster it needs to be able to create Routes and Secrets across namespaces.

[tnozicka]

And endpoints are needed cluster-wise https://github.com/tnozicka/openshift-acme/blob/master/deploy/clusterrole.yaml

[dustymabe]

@tnozicka i guess a good way to measure the 'permissions requirements' are to say: "Would it be possible to set this up in openshift online without any admin access". Is it possible for a user to put all of this stuff in his own namespace and grant access to the acmecontroller to just modify routes/services in that namespace.

[tnozicka]

@dustymabe yes, that's entirely possible with openshift-acme (one of the use cases I had in mind when writing it was actually an Online user with only his namespace)

[dustymabe]

@dustymabe yes, that's entirely possible with openshift-acme (one of the use cases I had in mind when writing it was actually an Online user with only his namespace)

@ibotty - does that detail solve any concerns you have? I still think writing out the user stories and verifying anything that ibotty/openshift-letsencrypt can do can be done by openshift-acme would be useful. Also after we make the user stories we should add documentation to make sure it is easy for users to set up. Thoughts?

[ibotty]

I am not happy with it needing access to any secrets. Apart from that, it looks like it covers my use cases. I will have to play with it first to be sure though. I am sure, it takes less resources than open shiftless encrypt, but I like to be certain :).

[tnozicka]

Yeah, I was expecting a question about Secrets :) So here is why we need those permissions: We store the user account key (and other information) for ACME (like Let's Encrypt) and all the certificates in Secrets so it needs permissions to list/create/update/delete them. With that said the controller will only list/create/update/delete secrets with a special annotation so it won't touch or view any other secrets that don't belong to it. But I don't think that can be expressed more fine grained with RBAC.

Also if you don't use Routes for your app and you Services get exposed by a LoadBalancer instead, the certificates have to be Secrets which get mounted into your pods so you can use those certificates when starting your servers.

And there are Ingresses that require you to store certs in Secrets, so you need those permissions there either way. Also last time I checked OpenShift should be slowly moving to Ingresses. Well, slowly seems to be the important word here :)

I think the resource usage should be pretty low, we don't do any pooling or stuff like that, it's all event based (WATCH).

[LorbusChris]

@tnozicka I am wondering how setup differs in the case when intended for use in one namespace only or with lacking permissions, e.g. on OpenShift Online.
For example, the template used in the role over in #25 cannot be used there as the user cannot create a cluster role.

[tnozicka]

@LorbusChris I am not an expert on permissions but I think you can just use Role instead of ClusterRole.

But let's try it from another end. The only thing that openshift-acme cares about it to be run with ServiceAccount that allows it to do what it needs. If you don't care about restricting permissions you can use for 1 namespace the default SA which is in every namespace (and is used by default if you don't specify any) and cluster-admin for all namespaces.

[LorbusChris]

On Online, deploying as a Role instead of a ClusterRole fails also, unfortunately 😭 would love to get this running.
Edit: Or do you mean I shoud simply remove the ClusterRole entirely for usage on Online?

[tnozicka]

@LorbusChris yeah, drop it. The role is just a constraint - it works the same without it.

[LorbusChris]

@tnozicka thanks for the support.
Now on to the next hurdle 😄
I'm getting these errors from the pod logs:

ERROR cmd.go:138 Unable to bootstrap certificate database: 'User "system:serviceaccount:acme-test:acme-controller" cannot list all secrets in the cluster'
ERROR cmd.go:173 Couln't initialize RouteController: 'RouteController could not find its own service: 'User "system:serviceaccount:acme-test:acme-controller" cannot get services in project "acme-test"''

Again that has to do with permissions...
Starting to think I should probably open a dedicated issue for OpenShift Online Support in this repo, as this is getting off topic.
In any way - I'll continue my trials to get this to run on Online!

[ibotty]

I think you won't have much luck. Openshift-acme will need some changes before it works on a rights constrained environment like openshift online. You will have better luck with openshift-letsencrypt. I do plan to migrate to this project as soon as these issues are resolved though. I plan to include a migration script though.

[tnozicka]

@LorbusChris yes, please open new issues so we can track those easily. I see that running this on Online could use some docs; it's in my queue now :)

My guess is that you didn't specify a namespace so that's why it's saying it can't list all secrets in a cluster - see this flag https://github.com/tnozicka/openshift-acme/blob/master/pkg/cmd/cmd.go#L94

[LorbusChris]

@tnozicka Unfortunately, specifying a namespace env variable didn't fix the problem.

I've created #28 to continue OpenShift Online related discusisons there

[cben]

Another question: how does this compare with https://github.com/jetstack/cert-manager/ ?

It already supports wildcards via plugins for a few cloud DNS providers: http://docs.cert-manager.io/en/latest/tutorials/acme/dns-validation.html?highlight=Wildcard
But I guess it's ignorant of Openshift routes?

[tnozicka]

Another question: how does this compare with https://github.com/jetstack/cert-manager/ ?

When I've started writing this it was kube-lego. I was considering if that project could be extended to support Routes, but it was requiring you to modify Ingress controller which was unacceptable. Not sure if they've changed it since. Also the fact that they are using CRDs is a dealbreaker since that is not multitenant and requires cluster-admin. It doesn't support Routes. (We will support also Ingress.)

[munnerz]

It doesn't support Routes. (We will support also Ingress.)

FWIW, we have plans in place that'll allow us to support alternative HTTP01 challenge solving strategies (e.g. Routes, Istio resources, etc.) cert-manager/cert-manager#783.

[vorburger]

ibotty/openshift-letsencrypt#31

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.