Native Kubernetes HPA in mimir-distributed

[jakubsikorski]

Is your feature request related to a problem? Please describe.

HPA support was added in as a response to issue #3430. The added HPA solution is based on KEDA, which requires additional operator to get installed on the cluster, with metrics to be present.

Describe the solution you'd like

It would be great to add HPA based on native Kubernetes provided metrics by the metrics-server, not relying on cadvisor metrics present in Prometheus, to mimir-distributed components (same components as for KEDA).

Describe alternatives you've considered

I've attempted to use the KEDA based HPA solution, however the complexity it adds due to additional CRs needed to use this solution is high.

[narqo]

My understanding is that one of the motivations to pick KEDA over the "bare bone" HPA on top of the metrics-server, was the requirement to scale, base on the Mimir's own metrics (e.g. refer to ScaledObject in the querier).

Projects, that allow exposing custom Prometheus metrics to metrics-server exist (kubernetes-sigs/prometheus-adapter), although replacing KEDA with different metrics-server adaptor, doesn't seem to help with the original issue of "extra complexity" after extra components.

That being said,

I've attempted to use the KEDA based HPA solution, however the complexity it adds due to additional CRs needed to use this solution is high.

the issue #7368 tracks the work left to promote KEDA-based autoscaling to stable. It could be helpful if you shared more details about your experience. Which parts currently contribute to the additional complexity?

[dimitarvdimitrov]

with similar arguments to this #7870 (comment) I think we should close this