Support filtering ingresses on `ingressClassName` as well as deprecated kubernetes.io/ingress.class annotation #2054

dsalisbury · 2021-04-18T06:36:47Z

Description

Adds the ability to filter ingresses based on their IngressClassName attribute. There's an --ingress-class-filter argument that's now exposed through the CLI and some additional filtering in source/ingress.go

Fixes #1975 #1792

Checklist

Unit tests updated
End user documentation updated

dsalisbury · 2021-04-18T06:45:47Z

~~So one thing I have outstanding is detecting the current default IngressClass (as per https://kubernetes.io/docs/concepts/services-networking/ingress/#default-ingress-class)~~

~~Here's my thinking:~~

~~in the setup of source/ingress.go, load the IngressClasses, find any with the special annotation, and store this away~~
~~the default class is used for any ingresses which are found and don't have a class explicitly~~
~~if more than 1 default ingress class is found then assume no default ingress?~~

Caution: If you have more than one IngressClass marked as the default for your cluster, the admission controller prevents creating new Ingress objects that don't have an ingressClassName specified. You can resolve this by ensuring that at most 1 IngressClass is marked as default in your cluster.
~~We probably need to watch for changes to the ingress classes and reconcile if/when the default changes~~
~~In an RBAC cluster, I think this will require some extra permissions so doc updates and things will be needed~~

~Any thoughts on this @Raffo? ~

seanmalloy · 2021-05-12T01:18:25Z

/kind feature

FalconerTC · 2021-09-22T16:41:56Z

Any updates here? Would really like to see this feature get through

wagnst · 2021-09-30T21:14:05Z

@dsalisbury can you fix the merge conflicts, maybe then someone will review?

...and update the help text to specify use more clearly

dsalisbury · 2021-10-02T05:25:30Z

Update on things:

detecting default ingress class isn't necessary; if there's a default and a new ingress is created with no class name set, the admission controller sets it explicitly.
code is updated and rebased on latest master

we don't want to get incompatible restrictions by letting someone set "--ingress-class" and --annotation-filter="kubernetes.io/ingress.class" at the same time

dsalisbury · 2021-10-02T06:40:46Z

I think this is ready for review!

I've done some manual verification locally with a kind cluster connected to pdns in the following scenarios:

two IngressClass defined (with one as default) and one Ingress of each class; checked with zero, one, and two --ingress-class args
Two IngressClass and neither is default, with an Ingress of each class
use of --ingress-class to match deprecated annotation-based ingress class specification
use of mixed ingress.class annotation and ingressClassName

All scenarios worked correctly as far as I could tell

szuecs · 2023-01-05T19:16:08Z

source/ingress.go

+		for _, nameFilter := range sc.ingressClassNames {
+			if ingress.Spec.IngressClassName != nil && nameFilter == *ingress.Spec.IngressClassName {
+				matched = true
+			} else if matchLabelSelector(selector, ingress.Annotations) {


this block does the filtering with fallback

szuecs · 2023-01-05T19:33:13Z

Maybe to add more context. I joined a couple of months ago to help shrinking the backlog.

@ryan-dyer-sp TBH I think you and other commenters show not a very nice community communication style. Naturally I would give more priority to people who are more friendly, because I don't want to spend time with unfriendly people.

You can just scroll the comments and see for example (others are not better and I don't want to pick one particular user here):

How hard could it be to review a PR this simple?

Examples like this make me re-thinking if I want to spend time maintaining this project, because it's not respectful IMO.

Please also understand that as a maintainer I am free to review any PR that I feel is interesting for the project. Maybe I looked at it already, but think it's wrong, but did not care because I don't use the feature and think the feature should be rather abandoned (just hypothetical not really true, because I am aware that people run multiple instances of external-dns).

KarstenSiemer · 2023-01-06T13:49:49Z

@szuecs thanks for your work inside the OSS world!
Lets not get disheartened. This is a wonderful project and it will keep being one.
I think OSS is all about respect from everyone and to everyone.

And someone used this project, tested it and has found something to improve and even submitted a PR, all for $0 and without even being a maintainer. Few people spend time on this problem and just felt left alone.
I'd say we should leave the personal stuff out of this and keep concentrated on the actual problem.

If there are any open tasks to get this feature to a production stage, I am also willing to implement.

szuecs · 2023-01-06T18:06:13Z

@KarstenSiemer thanks and I fully agree let's make it. Yesterday I added my questions, it's not that I have string opinions on them, but maybe it would be great to have some mixed config possible for migration purposes.
In other controller I first check ingressClass and then annotation, so ingressClass has precedence, so why do we check exclusive ingressClass vs. annotation?

johngmyers · 2023-01-08T05:15:59Z

docs/faq.md

+To do this with ExternalDNS you can use the `--ingress-class` to specifically tie an instance of ExternalDNS to
+an instance of a ingress controller. Let's assume you have two ingress controllers `nginx-internal` and `nginx-external`
+then you can start two ExternalDNS providers one with `--ingress-class=nginx-internal` and one with `--ingress-class=nginx-external`.


Suggested change

To do this with ExternalDNS you can use the `--ingress-class` to specifically tie an instance of ExternalDNS to

an instance of a ingress controller. Let's assume you have two ingress controllers `nginx-internal` and `nginx-external`

then you can start two ExternalDNS providers one with `--ingress-class=nginx-internal` and one with `--ingress-class=nginx-external`.

To do this with ExternalDNS you can use the `--ingress-class` flag to specifically tie an instance of ExternalDNS to

an instance of a ingress controller. Let's assume you have two ingress controllers, `nginx-internal` and `nginx-external`.

You can then start two ExternalDNS providers, one with `--ingress-class=nginx-internal` and one with `--ingress-class=nginx-external`.

johngmyers · 2023-01-08T05:17:36Z

docs/faq.md


-Beware when using multiple sources, e.g. `--source=service --source=ingress`, `--annotation-filter` will filter every given source objects.
-If you need to filter only one specific source you have to run a separated external dns service containing only the wanted `--source`  and `--annotation-filter`.
+The `--ingress-class` argument will check both the `ingressClassName` field as well as the deprecated `kubernetes.io/ingress.class` annotation.


Suggested change

The `--ingress-class` argument will check both the `ingressClassName` field as well as the deprecated `kubernetes.io/ingress.class` annotation.

The `--ingress-class` flag will check both the `ingressClassName` field and the deprecated `kubernetes.io/ingress.class` annotation.

“argument” is preferable. Golang calls command line arguments “flags” whether the type is a boolean or not, but the rest of the world of IT typically uses “flag” just for booleans (maybe tristates).

johngmyers · 2023-01-08T05:18:33Z

docs/faq.md


-**Note:** Filtering based on annotation means that the external-dns controller will receive all resources of that kind and then filter on the client-side.
+Note: the `--ingress-class` argument cannot be used at the same time as a `kubernetes.io/ingress.class` annotation filter; if you do this an error will be raised.


Suggested change

Note: the `--ingress-class` argument cannot be used at the same time as a `kubernetes.io/ingress.class` annotation filter; if you do this an error will be raised.

Note: the `--ingress-class` flag cannot be used at the same time as a `kubernetes.io/ingress.class` annotation filter; if you do this an error will be raised.

See https://github.com/kubernetes-sigs/external-dns/pull/2054/files#r1071303410

johngmyers · 2023-01-08T05:25:51Z

source/ingress.go

+// class
+func (sc *ingressSource) filterByIngressClass(ingresses []*networkv1.Ingress) ([]*networkv1.Ingress, error) {
+	// if no class filter is specified then there's nothing to do
+	if sc.ingressClassNames == nil {


Suggested change

if sc.ingressClassNames == nil {

if len(sc.ingressClassNames) == 0 {

johngmyers · 2023-01-08T05:31:05Z

source/ingress.go

+
+		requirements, _ := selector.Requirements()
+		for _, requirement := range requirements {
+			if requirement.Key() == "kubernetes.io/ingress.class" {


filterByIngressClass() does the fallback logic when considering whether an Ingress matches the requirement set by the --ingress-class flag. For that reason, passing both an --ingress-class flag and a kubernetes.io/ingress.class annotation filter doesn't make sense: it would exclude those ingresses that don't match the annotation filter. Thus, this check is a reasonable validation against the admin passing nonsensical flag combinations.

johngmyers · 2023-01-08T05:37:23Z

source/ingress_test.go

+					annotations: map[string]string{
+						"kubernetes.io/ingress.class": "dmz",
+					},
+				},


Suggest test case with annotation dmz and ingressClassName internal. Should not be in expected.

Possibly also annotation internal ingressClassName dmz. Should be in expected.

Suggest test case with neither annotation nor ingressClassName. Should not be in expected.

k8s-ci-robot · 2023-01-08T05:38:49Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dsalisbury
Once this PR has been reviewed and has the lgtm label, please ask for approval from njuettner by writing /assign @njuettner in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sftim

Does this need to change too? It mentions kubernetes.io/ingress.class

Additionally, you may leverage [cert-manager](https://github.com/jetstack/cert-manager) to automatically issue SSL certificates from [Let's Encrypt](https://letsencrypt.org/). To do that, request a certificate in public service definition:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/acme-challenge-type: "dns01"
    certmanager.k8s.io/acme-dns01-provider: "route53"
    certmanager.k8s.io/cluster-issuer: "letsencrypt-production"
    kubernetes.io/ingress.class: "external-ingress"
    kubernetes.io/tls-acme: "true"

sftim

Does the FAQ question I have a Service/Ingress but it's ignored by ExternalDNS. Why?“ need a new answer?

sftim · 2023-01-16T14:29:47Z

source/ingress.go

+		return ingresses, nil
+	}
+
+	classNameReq, err := labels.NewRequirement("kubernetes.io/ingress.class", selection.In, sc.ingressClassNames)


Labels? Is that right? The deprecated kubernetes.io/ingress.class annotation is an annotation, not a label.

sftim · 2023-01-16T14:34:50Z

docs/faq.md

+However, beware when using annotation filters with multiple sources, e.g. `--source=service --source=ingress`, since `--annotation-filter` will filter every given source objects.
+If you need to use annotation filters against a specific source you have to run a separated external dns service containing only the wanted `--source`  and `--annotation-filter`.
+
+**Note:** Filtering based on annotation or ingress class name means that the external-dns controller will receive all resources of that kind and then filter on the client-side.
 In larger clusters with many resources which change frequently this can cause performance issues. If only some resources need to be managed by an instance
 of external-dns then label filtering can be used instead of annotation filtering. This means that only those resources which match the selector specified


Suggested change

of external-dns then label filtering can be used instead of annotation filtering. This means that only those resources which match the selector specified

of external-dns then label filtering can be used instead of ingress class filtering (or legacy annotation filtering). This means that only those resources which match the selector specified

?

hdiass · 2023-04-05T12:00:01Z

whats preventing this PR to be merged ?

mateuszdrab · 2023-04-05T12:27:47Z

whats preventing this PR to be merged ?

Not sure, this PR is notoriously ignored by the maintainers.

I made a workaround that works and fits into my design well so this feature is not needed so critically but would still be useful to address a few edge cases.

hdiass · 2023-04-05T12:42:44Z

how did you worked this around without annotations ?

mateuszdrab · 2023-04-05T14:48:02Z

how did you worked this around without annotations ?

It was with annotations, but not the ingress.class one. Since my external facing records are cnames, I set external DNS to only act when the target annotation is in place. Still, it would be more convenient to just scope it to one ingress class.

hdiass · 2023-04-05T16:59:54Z

Yeah i did the same, but is annoying when you already have that meta in the ingressClassName

szuecs · 2023-04-05T18:31:00Z

There are more than a couple of suggestions to change by 2 different members and as a maintainer I wait such that these are addressed.
@mateuszdrab please be careful what you write, because it's wrong and you create just hate to us, which is not fine.

mateuszdrab · 2023-04-05T19:56:58Z

There are more than a couple of suggestions to change by 2 different members and as a maintainer I wait such that these are addressed.
@mateuszdrab please be careful what you write, because it's wrong and you create just hate to us, which is not fine.

I understand and I feel bad for making it sound like this. It is just how I feel having seen someone put the effort into making this PR and I'd have thought the maintainers are here to drive progression and not just approve/deny changes.

A lot of conversation is about ethics and little is done to progress with merging this. I don't like political discussion so I generally don't comment when unnecessary. It sort of appears to me that the maintainers get defensive as soon as people start asking for attention on this PR.

I see new releases come out without this feature and it's been a long time since this was opened - it personally doesn't affect me as I only run this in a home lab. From a perspective of other users, it could be more significant.

However, I have a right to express my opinion, and my opinion is that bugs/issues should be addressed before new features and deprecation of the ingress annotation is quite a fundamental change, a breaking change for some people who might be heavily dependent on external DNS and filtering by ingress classes. Kubernetes will not allow you to have both ingressclassname and the annotation in place and therefore the lack of support here could be potentially a blocker in migration to higher versions forcing some to stay on unsupported versions. We're now on 1.27 almost and this was deprecated on 1.18 so almost 10 releases ago...

If I was the person who made a PR and couldn't get it merged for 2 years despite it being realtively small in size I'd just give up and it might be why the OP isn't responding.

szuecs · 2023-04-05T21:52:06Z

As I wrote the pr was reviewed by two kubernetes contributors and there was no change and you did just bike shedding @mateuszdrab which is not ok.

mateuszdrab · 2023-04-05T22:38:33Z

I understand that the PR has been reviewed but let me highlight one fact that it appears that @dsalisbury has not made any contributions on GitHub for the past 6 months.
Community members are asking for progress and it's all stalled because one person is not engaging. We don't know if he will come back here, but we do know this PR is needed - what can be done to make progress on this?

Can we make suggestions to his code and you maintainers can merge those in? (did he allow maintainer edits?)
Do we need someone else to take ownership of this code and drive towards merging?
Do we leave it to stale out and die only for someone else to open an issue on it again - this does not sound like a good option, does it?

szuecs · 2023-04-06T06:31:41Z

@mateuszdrab take ownership if you want to have the pr merged and commit the changes then we will review the pr as people show here interest.

dsalisbury · 2023-04-24T08:39:03Z

I understand that the PR has been reviewed but let me highlight one fact that it appears that @dsalisbury has not made any contributions on GitHub for the past 6 months.
Community members are asking for progress and it's all stalled because one person is not engaging. We don't know if he will come back here, but we do know this PR is needed - what can be done to make progress on this?

Yep hi, that's me. Apologies for lack of engagement, but I don't have time or capacity to see this through any more. Anyone who wants is welcome to take the branch and run with it themselves or just throw it away and start over.

k8s-ci-robot · 2023-04-25T06:30:27Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

alefray · 2023-05-04T14:22:26Z

Yep hi, that's me. Apologies for lack of engagement, but I don't have time or capacity to see this through any more. Anyone who wants is welcome to take the branch and run with it themselves or just throw it away and start over.

Thanks to @dsalisbury and to all the reviewers for their work 🙏

I've commit the changes asked in #3589

I hope we can finish this 😉 !

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 18, 2021

k8s-ci-robot requested review from njuettner and Raffo April 18, 2021 06:37

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 18, 2021

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label May 12, 2021

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 14, 2021

dsalisbury added 7 commits October 2, 2021 14:29

Plumb in filtering on ingress class name

c4898b7

Add ingressClassNameFilter testing

115e250

ingress source: update code for networkv1

c823eba

cli args: rename ingress-class-filter to ingress-classes

0b6c67f

ingress source: improve class name filtering and logging

da71b3f

Rename ingressClassNameFilter to ingressClassNames

8da6f99

...and update the help text to specify use more clearly

Update tests for ingress class filtering

ac0c4be

dsalisbury added 3 commits October 2, 2021 16:09

ingress source: check for duplicate classname configs

71a672f

we don't want to get incompatible restrictions by letting someone set "--ingress-class" and --annotation-filter="kubernetes.io/ingress.class" at the same time

ingress source: ingressClassNames now feed into annotation filter

901effb

docs: add info about new --ingress-class

807e213

dsalisbury force-pushed the ingress-class-filtering branch from 4b1e521 to 807e213 Compare October 2, 2021 06:37

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 2, 2021

dsalisbury marked this pull request as ready for review October 2, 2021 06:40

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 2, 2021

docs: format ingressClassName as inline code

8ec342a

szuecs reviewed Jan 5, 2023

View reviewed changes

johngmyers suggested changes Jan 8, 2023

View reviewed changes

aojea mentioned this pull request Jan 16, 2023

Cannot create ingress with ingressClass annotation and IngressClassName both set kubernetes/kubernetes#115018

Closed

sftim reviewed Jan 16, 2023

View reviewed changes

mateuszdrab mentioned this pull request Feb 6, 2023

Allow global configuration of TTL #1829

Open

gw0 mentioned this pull request Mar 7, 2023

The reliance on deprecated annotation causes Kubernetes validation issues #2386

Closed

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 25, 2023

alefray mentioned this pull request May 4, 2023

(V2) Support filtering ingresses on ingressClassName as well as deprecated kubernetes.io/ingress.class annotation #3589

Merged

2 tasks

k8s-ci-robot merged commit f76382a into kubernetes-sigs:master May 9, 2023

johngmyers mentioned this pull request Jun 7, 2023

Add johngmyers to reviewers #3664

Merged

2 tasks

gw0 mentioned this pull request Jun 28, 2023

[bitnami/external-dns] New version 0.13.5 available upstream bitnami/charts#17382

Closed

johngmyers mentioned this pull request Sep 13, 2023

Add johngmyers as approver #3931

Merged

2 tasks

	The `--ingress-class` argument will check both the `ingressClassName` field as well as the deprecated `kubernetes.io/ingress.class` annotation.
	The `--ingress-class` flag will check both the `ingressClassName` field and the deprecated `kubernetes.io/ingress.class` annotation.


		Note: Filtering based on annotation means that the external-dns controller will receive all resources of that kind and then filter on the client-side.
		Note: the `--ingress-class` argument cannot be used at the same time as a `kubernetes.io/ingress.class` annotation filter; if you do this an error will be raised.

	Note: the `--ingress-class` argument cannot be used at the same time as a `kubernetes.io/ingress.class` annotation filter; if you do this an error will be raised.
	Note: the `--ingress-class` flag cannot be used at the same time as a `kubernetes.io/ingress.class` annotation filter; if you do this an error will be raised.

	if sc.ingressClassNames == nil {
	if len(sc.ingressClassNames) == 0 {

	of external-dns then label filtering can be used instead of annotation filtering. This means that only those resources which match the selector specified
	of external-dns then label filtering can be used instead of ingress class filtering (or legacy annotation filtering). This means that only those resources which match the selector specified

Support filtering ingresses on ingressClassName as well as deprecated kubernetes.io/ingress.class annotation #2054

Support filtering ingresses on ingressClassName as well as deprecated kubernetes.io/ingress.class annotation #2054

Conversation

dsalisbury commented Apr 18, 2021 • edited Loading

dsalisbury commented Apr 18, 2021 • edited Loading

seanmalloy commented May 12, 2021

FalconerTC commented Sep 22, 2021

wagnst commented Sep 30, 2021

dsalisbury commented Oct 2, 2021

dsalisbury commented Oct 2, 2021 • edited Loading

Choose a reason for hiding this comment

szuecs commented Jan 5, 2023

KarstenSiemer commented Jan 6, 2023

szuecs commented Jan 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jan 8, 2023

sftim left a comment

Choose a reason for hiding this comment

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hdiass commented Apr 5, 2023

mateuszdrab commented Apr 5, 2023

hdiass commented Apr 5, 2023

mateuszdrab commented Apr 5, 2023

hdiass commented Apr 5, 2023

szuecs commented Apr 5, 2023

mateuszdrab commented Apr 5, 2023 • edited Loading

szuecs commented Apr 5, 2023

mateuszdrab commented Apr 5, 2023 • edited Loading

szuecs commented Apr 6, 2023

dsalisbury commented Apr 24, 2023

k8s-ci-robot commented Apr 25, 2023

alefray commented May 4, 2023

Support filtering ingresses on `ingressClassName` as well as deprecated kubernetes.io/ingress.class annotation #2054

Support filtering ingresses on `ingressClassName` as well as deprecated kubernetes.io/ingress.class annotation #2054

dsalisbury commented Apr 18, 2021 •

edited

Loading

dsalisbury commented Apr 18, 2021 •

edited

Loading

dsalisbury commented Oct 2, 2021 •

edited

Loading

mateuszdrab commented Apr 5, 2023 •

edited

Loading

mateuszdrab commented Apr 5, 2023 •

edited

Loading