Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update readme #5

Merged
merged 3 commits into from
Aug 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 33 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
This is a meta-monitoring chart for GEL, GEM and GET. It should be installed in a
separate namespace next to GEM, GEL or GET installations.

Note that this is pre-production software at the moment.

## Preparation

Create a values.yaml file based on the [default one](../charts/meta-monitoring/values.yaml).
Expand All @@ -15,29 +17,54 @@ Create a values.yaml file based on the [default one](../charts/meta-monitoring/v

## Local and cloud modes

The chart has 2 modes: local and cloud. In the local mode logs, metrics and traces are sent
The chart has 2 modes: local and cloud. In the local mode logs, metrics and/or traces are sent
to small Loki, Mimir and Tempo installations running in the meta-monitoring namespace.

![local mode](docs/images/Meta%20monitoring%20local.png)

To enable local mode set `local.enabled` to true.
To enable local mode set `local.<logs|metrics|traces>.enabled` to true.

In the cloud mode the logs, metrics and traces are sent to
In the cloud mode the logs, metrics and/or traces are sent to Grafana Cloud.

![cloud mode](docs/images/Meta%20monitoring%20cloud.png)

To enable cloud mode set `cloud.enabled` to true. The `endpoint`, `username` and `password` settings for your Grafana Cloud logs, metrics and traces instances have to be filled in as well.
To enable cloud mode set `cloud.<logs|metrics|traces>.enabled` to true. The `endpoint`, `username` and `password` settings for your Grafana Cloud logs, metrics and traces instances have to be filled in as well.

Both modes can be enabled at the same time.

## Installation

```
helm install -n meta -f values.yaml meta ./charts/meta-monitoring
helm install -n meta --skip-crds -f values.yaml meta ./charts/meta-monitoring
```

If the platform supports CRDs the `--skip-crds` option can be removed. However the CRDs are not used by this chart.

For more instructions including how to update the chart go to the [installation](docs/installation.md) page.

## Supported features

- Specify which namespaces are monitored
- Specify if logs, metrics or traces should be enabled for cloud or local
- Specify the cluster name used for the logs, metrics and traces
- Specify PII regexes that are applied to logs before they are sent to Loki (cloud or local). The capture group in the regex is replaced with *****.
- a Grafana instance is installed (when local mode is used) with the relevant datasources installed. The following dashboards are installed:
- logs dashboards
- metrics dashboards
- traces dashboards
- agent dashboards
- Retention is set to 24 hours

Most of these features are enabled by default. See the values.yaml file for how to enable/disable them.

## Caveats

- The [loki.source.kubernetes](https://grafana.com/docs/agent/latest/flow/reference/components/loki.source.kubernetes/) component of the Grafana Agent is used to scrape Kubernetes log files. This component is marked experimental at the moment.
- This has not been tested on Openshift yet.
- The underlying Loki, Mimir and Tempo are at the default size installed by the Helm chart. This might need changing when monitoring bigger Loki, Mimir or Tempo installations.
- MinIO is used as storage at the moment with a limited retention. At the moment this chart cannot be used for monitoring over longer periods.
- Agent self monitoring is not done at the moment.

## Developer help topics

- [update dependencies](docs/dev_update_dependencies.md)
- [update dependencies](docs/dev_update_dependencies.md)
10 changes: 7 additions & 3 deletions charts/meta-monitoring/values.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
# Specify the namespaces to monitor here
namespacesToMonitor:
- loki
- mimir
- tempo
clusterName: "meta-monitoring" # TODO check if this can be derived
# The name of the cluster where this will be installed
clusterName: "meta-monitoring"

# Set to true for a local version of logs, metrics or traces
local:
logs:
enabled: false
Expand All @@ -14,6 +17,7 @@ local:
minio:
enabled: false # This should be set to true if any of the previous is enabled

# Set to true to write logs, metrics or traces to Grafana Cloud
cloud:
logs:
enabled: true
Expand All @@ -31,7 +35,7 @@ cloud:
username:
password:

# Adding regexes here will add a stage.replace block. For more information see
# Adding regexes here will add a stage.replace block for logs. For more information see
# https://grafana.com/docs/agent/latest/flow/reference/components/loki.process/#stagereplace-block
logs:
piiRegexes:
Expand Down Expand Up @@ -63,7 +67,7 @@ kubeStateMetrics:
endpoint: kube-state-metrics.kube-state-metrics.svc.cluster.local:8080

# The following are configuration for the dependencies.
# These should not be changed.
# These should usually not be changed.

loki:
loki:
Expand Down