[Improvement] Improve performance with large number of secrets/configmaps. Strain on kube-apiserver

[jwierzbo]

On our test cluster we have 2500 Secrets and 4000 ConfigMaps objects.
Once kubernetes-reflector has been installed/started, kube-apiserver needs many times more resources then before:

In the logs of kube-apiserver I've observed tons of requests from kubernetes-reflector.
Also kubernetes-reflector required pretty much resources (2 GB of memory).

Is there any way to reduce such a load on kube-apiserver (e.g. define the list of monitored namespaces, extend the monitoring interval, etc.)?

[winromulus]

@jwierzbo what version of reflector are you using? Normally reflector should be limited to 256MB max (in the default chart)
As for limiting reflector there are no options right now, but I can put that on my list for this or next weekend.

[jwierzbo]

I'm using newest version 5.4.17.

I've tested it also on different, smaller cluster with 400 Secrets and 100 ConfigMaps and in that case 1GB of memory was sufficient

[stale]

Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Automatically closed stale item.

[k-ayache]

hi,
I have exactly the same issue with the same version 5.4.17, can someone give more details please about that

[stv0g]

Me too

[cwrau]

We're having the same problem, APIServer using more than 2 Cores and more than 200MiB/s traffic

Without reflector, it's using 0.5 Cores and 25MiB/s

Our cluster is maybe medium size

[mshade]

Us, too - etcd and k8s apiserver get a prohibitive amount of traffic caused by the reflector.