Create centralized prometheus

[sduranc]

We plan to set a centralized storage for prometheus metrics. This is meant to be used to report across all Altinn products. There are options to consider like Mimir, Thanos. We will try out Azure hosted prometheus first to see if that give us the features we need.

Tasks

Preview Give feedback

1. Deploy Azure Monitor Workspace
2. Look at how to gather data from differenet sources

How to send metrics

• https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/prometheus-remote-write-virtual-machines?tabs=managed-identity%2Cprom-operator

Findings

• there is a 1-to-1 relationship between AKS and AMW. So using the ama-metrics agent to send to 2 different amw is not an option.
• the best way probably is to remote write to amw with an agent.