Skip to content

Commit 6dc552f

Browse files
feat: Updated SS and microservices deployment docs (#13083)
(cherry picked from commit 1b80458)
1 parent 59a96cb commit 6dc552f

File tree

2 files changed

+582
-33
lines changed

2 files changed

+582
-33
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,346 @@
1+
---
2+
title: Install the microservice Helm chart
3+
menuTitle: Install microservice Loki
4+
description: Installing Loki in microservice (distributed) mode using the Helm chart.
5+
weight: 300
6+
keywords:
7+
---
8+
9+
# Install the microservice Helm chart
10+
11+
This Helm Chart deploys Grafana Loki on Kubernetes.
12+
13+
This chart configures Loki to run Loki in [microservice / distributed mode]({{< relref "../../../../get-started/deployment-modes#microservices-mode" >}}). The microservices deployment mode runs components of Loki as distinct processes.
14+
15+
The default Helm chart deploys the following components:
16+
- **Compactor component** (1 replica): Compacts and processes stored data.
17+
- **Distributor component** (3 replicas, maxUnavailable: 2): Distributes incoming requests. Up to 2 replicas can be unavailable during updates.
18+
- **IndexGateway component** (2 replicas, maxUnavailable: 1): Handles indexing. Up to 1 replica can be unavailable during updates.
19+
- **Ingester component** (3 replicas): Handles ingestion of data.
20+
- **Querier component** (3 replicas, maxUnavailable: 2): Processes queries. Up to 2 replicas can be unavailable during updates.
21+
- **QueryFrontend component** (2 replicas, maxUnavailable: 1): Manages frontend queries. Up to 1 replica can be unavailable during updates.
22+
- **QueryScheduler component** (2 replicas): Schedules queries.
23+
24+
It is not recommended to run scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example.
25+
26+
**Prerequisites**
27+
28+
- Helm 3 or above. See [Installing Helm](https://helm.sh/docs/intro/install/).
29+
- A running Kubernetes cluster.
30+
- (Optional) A Memcached deployment for better query performance. For information on configuring Memcached, refer to the [caching section](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/caching/).
31+
32+
33+
**To deploy Loki in microservice mode (with MinIO):**
34+
35+
36+
1. Add [Grafana's chart repository](https://github.com/grafana/helm-charts) to Helm:
37+
38+
```bash
39+
helm repo add grafana https://grafana.github.io/helm-charts
40+
```
41+
42+
2. Update the chart repository:
43+
44+
```bash
45+
helm repo update
46+
```
47+
48+
3. Create the configuration file `values.yaml`. The example below illustrates how to deploy Loki in test mode using MinIO as storage:
49+
50+
```yaml
51+
loki:
52+
schemaConfig:
53+
configs:
54+
- from: 2024-04-01
55+
store: tsdb
56+
object_store: s3
57+
schema: v13
58+
index:
59+
prefix: loki_index_
60+
period: 24h
61+
ingester:
62+
chunk_encoding: snappy
63+
tracing:
64+
enabled: true
65+
querier:
66+
# Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing
67+
max_concurrent: 4
68+
69+
#gateway:
70+
# ingress:
71+
# enabled: true
72+
# hosts:
73+
# - host: FIXME
74+
# paths:
75+
# - path: /
76+
# pathType: Prefix
77+
78+
deploymentMode: Distributed
79+
80+
ingester:
81+
replicas: 3
82+
querier:
83+
replicas: 3
84+
maxUnavailable: 2
85+
queryFrontend:
86+
replicas: 2
87+
maxUnavailable: 1
88+
queryScheduler:
89+
replicas: 2
90+
distributor:
91+
replicas: 3
92+
maxUnavailable: 2
93+
compactor:
94+
replicas: 1
95+
indexGateway:
96+
replicas: 2
97+
maxUnavailable: 1
98+
99+
bloomCompactor:
100+
replicas: 0
101+
bloomGateway:
102+
replicas: 0
103+
104+
# Enable minio for storage
105+
minio:
106+
enabled: true
107+
108+
# Zero out replica counts of other deployment modes
109+
backend:
110+
replicas: 0
111+
read:
112+
replicas: 0
113+
write:
114+
replicas: 0
115+
116+
singleBinary:
117+
replicas: 0
118+
```
119+
120+
4. Install or upgrade the Loki deployment.
121+
- To install:
122+
```bash
123+
helm install --values values.yaml loki grafana/loki
124+
```
125+
- To upgrade:
126+
```bash
127+
helm upgrade --values values.yaml loki grafana/loki
128+
```
129+
130+
131+
1. Verify that Loki is running:
132+
```bash
133+
kubectl get pods -n loki
134+
```
135+
The output should an output similar to the following:
136+
137+
```bash
138+
loki-canary-8thrx 1/1 Running 0 167m
139+
loki-canary-h965l 1/1 Running 0 167m
140+
loki-canary-th8kb 1/1 Running 0 167m
141+
loki-chunks-cache-0 0/2 Pending 0 167m
142+
loki-compactor-0 1/1 Running 0 167m
143+
loki-compactor-1 1/1 Running 0 167m
144+
loki-distributor-7c9bb8f4dd-bcwc5 1/1 Running 0 167m
145+
loki-distributor-7c9bb8f4dd-jh9h8 1/1 Running 0 167m
146+
loki-distributor-7c9bb8f4dd-np5dw 1/1 Running 0 167m
147+
loki-gateway-77bc447887-qgc56 1/1 Running 0 167m
148+
loki-index-gateway-0 1/1 Running 0 167m
149+
loki-index-gateway-1 1/1 Running 0 166m
150+
loki-ingester-zone-a-0 1/1 Running 0 167m
151+
loki-ingester-zone-b-0 1/1 Running 0 167m
152+
loki-ingester-zone-c-0 1/1 Running 0 167m
153+
loki-minio-0 1/1 Running 0 167m
154+
loki-querier-bb8695c6d-bv9x2 1/1 Running 0 167m
155+
loki-querier-bb8695c6d-bz2rw 1/1 Running 0 167m
156+
loki-querier-bb8695c6d-z9qf8 1/1 Running 0 167m
157+
loki-query-frontend-6659566b49-528j5 1/1 Running 0 167m
158+
loki-query-frontend-6659566b49-84jtx 1/1 Running 0 167m
159+
loki-query-frontend-6659566b49-9wfr7 1/1 Running 0 167m
160+
loki-query-scheduler-f6dc4b949-fknfk 1/1 Running 0 167m
161+
loki-query-scheduler-f6dc4b949-h4nwh 1/1 Running 0 167m
162+
loki-query-scheduler-f6dc4b949-scfwp 1/1 Running 0 167m
163+
loki-results-cache-0 2/2 Running 0 167m
164+
```
165+
166+
## Object Storage Configuration
167+
168+
After testing Loki with MinIO, it is recommended to configure Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers:
169+
170+
{{< code >}}
171+
172+
```s3
173+
# Example configuration for Loki with S3 storage
174+
175+
loki:
176+
schemaConfig:
177+
configs:
178+
- from: 2024-04-01
179+
store: tsdb
180+
object_store: s3
181+
schema: v13
182+
index:
183+
prefix: loki_index_
184+
period: 24h
185+
ingester:
186+
chunk_encoding: snappy
187+
tracing:
188+
enabled: true
189+
querier:
190+
max_concurrent: 4
191+
192+
storage:
193+
type: s3
194+
bucketNames:
195+
chunks: "chunks"
196+
ruler: "ruler"
197+
admin: "admin"
198+
s3:
199+
# s3 URL can be used to specify the endpoint, access key, secret key, and bucket name
200+
s3: s3://access_key:secret_access_key@custom_endpoint/bucket_name
201+
# AWS endpoint URL
202+
endpoint: <your-endpoint>
203+
# AWS region where the S3 bucket is located
204+
region: <your-region>
205+
# AWS secret access key
206+
secretAccessKey: <your-secret-access-key>
207+
# AWS access key ID
208+
accessKeyId: <your-access-key-id>
209+
# AWS signature version (e.g., v2 or v4)
210+
signatureVersion: <your-signature-version>
211+
# Forces the path style for S3 (true/false)
212+
s3ForcePathStyle: false
213+
# Allows insecure (HTTP) connections (true/false)
214+
insecure: false
215+
# HTTP configuration settings
216+
http_config: {}
217+
218+
deploymentMode: Distributed
219+
220+
# Disable minio storage
221+
minio:
222+
enabled: false
223+
224+
ingester:
225+
replicas: 3
226+
querier:
227+
replicas: 3
228+
maxUnavailable: 2
229+
queryFrontend:
230+
replicas: 2
231+
maxUnavailable: 1
232+
queryScheduler:
233+
replicas: 2
234+
distributor:
235+
replicas: 3
236+
maxUnavailable: 2
237+
compactor:
238+
replicas: 1
239+
indexGateway:
240+
replicas: 2
241+
maxUnavailable: 1
242+
243+
bloomCompactor:
244+
replicas: 0
245+
bloomGateway:
246+
replicas: 0
247+
248+
backend:
249+
replicas: 0
250+
read:
251+
replicas: 0
252+
write:
253+
replicas: 0
254+
255+
singleBinary:
256+
replicas: 0
257+
258+
```
259+
260+
```azure
261+
# Example configuration for Loki with Azure Blob Storage
262+
263+
loki:
264+
schemaConfig:
265+
configs:
266+
- from: 2024-04-01
267+
store: tsdb
268+
object_store: azure
269+
schema: v13
270+
index:
271+
prefix: loki_index_
272+
period: 24h
273+
ingester:
274+
chunk_encoding: snappy
275+
tracing:
276+
enabled: true
277+
querier:
278+
max_concurrent: 4
279+
280+
storage:
281+
type: azure
282+
azure:
283+
# Name of the Azure Blob Storage account
284+
accountName: <your-account-name>
285+
# Key associated with the Azure Blob Storage account
286+
accountKey: <your-account-key>
287+
# Comprehensive connection string for Azure Blob Storage account (Can be used to replace endpoint, accountName, and accountKey)
288+
connectionString: <your-connection-string>
289+
# Flag indicating whether to use Azure Managed Identity for authentication
290+
useManagedIdentity: false
291+
# Flag indicating whether to use a federated token for authentication
292+
useFederatedToken: false
293+
# Client ID of the user-assigned managed identity (if applicable)
294+
userAssignedId: <your-user-assigned-id>
295+
# Timeout duration for requests made to the Azure Blob Storage account (in seconds)
296+
requestTimeout: <your-request-timeout>
297+
# Domain suffix of the Azure Blob Storage service endpoint (e.g., core.windows.net)
298+
endpointSuffix: <your-endpoint-suffix>
299+
bucketNames:
300+
chunks: "chunks"
301+
ruler: "ruler"
302+
admin: "admin"
303+
deploymentMode: Distributed
304+
305+
ingester:
306+
replicas: 3
307+
querier:
308+
replicas: 3
309+
maxUnavailable: 2
310+
queryFrontend:
311+
replicas: 2
312+
maxUnavailable: 1
313+
queryScheduler:
314+
replicas: 2
315+
distributor:
316+
replicas: 3
317+
maxUnavailable: 2
318+
compactor:
319+
replicas: 1
320+
indexGateway:
321+
replicas: 2
322+
maxUnavailable: 1
323+
324+
bloomCompactor:
325+
replicas: 0
326+
bloomGateway:
327+
replicas: 0
328+
329+
backend:
330+
replicas: 0
331+
read:
332+
replicas: 0
333+
write:
334+
replicas: 0
335+
336+
singleBinary:
337+
replicas: 0
338+
339+
```
340+
{{< /code >}}
341+
342+
To configure other storage providers, refer to the [Helm Chart Reference]({{< relref "../reference" >}}).
343+
344+
## Next Steps
345+
* Configure an agent to [send log data to Loki](/docs/loki/<LOKI_VERSION>/send-data/).
346+
* Monitor the Loki deployment using the [Meta Monitoring Healm chart](/docs/loki/<LOKI_VERSION>/setup/install/helm/monitor-and-alert/)

0 commit comments

Comments
 (0)