You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: docs/sources/send-data/_index.md
+6-6
Original file line number
Diff line number
Diff line change
@@ -20,13 +20,13 @@ The following clients are developed and supported (for those customers who have
20
20
21
21
-[Grafana Alloy](https://grafana.com/docs/alloy/latest/) - Grafana Alloy is a vendor-neutral distribution of the OpenTelemetry (OTel) Collector. Alloy offers native pipelines for OTel, Prometheus, Pyroscope, Loki, and many other metrics, logs, traces, and profile tools. In addition, you can use Alloy pipelines to do different tasks, such as configure alert rules in Loki and Mimir. Alloy is fully compatible with the OTel Collector, Prometheus Agent, and Promtail. You can use Alloy as an alternative to either of these solutions or combine it into a hybrid system of multiple collectors and agents. You can deploy Alloy anywhere within your IT infrastructure and pair it with your Grafana LGTM stack, a telemetry backend from Grafana Cloud, or any other compatible backend from any other vendor.
-[Grafana Agent](/docs/agent/latest/) - The Grafana Agent is a client for the Grafana stack. It can collect telemetry data for metrics, logs, traces, and continuous profiles and is fully compatible with the Prometheus, OpenTelemetry, and Grafana open source ecosystems.
24
-
-[Promtail](https://grafana.com/docs/loki/<LOKI_VERSION>/send-data/promtail/) - Promtail can be configured to automatically scrape logs from Kubernetes pods running on the same node that Promtail runs on. Promtail and Prometheus running together in Kubernetes enables powerful debugging: if Prometheus and Promtail use the same labels, users can use tools like Grafana to switch between metrics and logs based on the label set. Promtail can be configured to tail logs from all files given a host path. It is the easiest way to send logs to Loki from plain-text files (for example, things that log to `/var/log/*.log`).
25
-
Promtail works well if you want to extract metrics from logs such as counting the occurrences of a particular message.
26
-
{{< admonition type="note" >}}
27
-
Promtail is feature complete. All future feature development will occur in Grafana Alloy.
28
-
{{< /admonition >}}
29
23
-[xk6-loki extension](https://github.com/grafana/xk6-loki) - The k6-loki extension lets you perform [load testing on Loki](https://grafana.com/docs/loki/<LOKI_VERSION>/send-data/k6/).
24
+
-[Grafana Agent](/docs/agent/latest/) (DEPRECATED) - The Grafana Agent is a client for the Grafana stack. It can collect telemetry data for metrics, logs, traces, and continuous profiles and is fully compatible with the Prometheus, OpenTelemetry, and Grafana open source ecosystems.
25
+
-[Promtail](https://grafana.com/docs/loki/<LOKI_VERSION>/send-data/promtail/) (DEPRECATED) - Promtail can be configured to automatically scrape logs from Kubernetes pods running on the same node that Promtail runs on.
26
+
{{< admonition type="caution" >}}
27
+
Promtail is deprecated. If you are currently using Promtail, you should plan your [migration to Alloy](https://grafana.com/docs/loki/<LOKI_VERSION>/setup/migrate/migrate-to-alloy/). All future feature development will occur in Grafana Alloy.
Copy file name to clipboardexpand all lines: docs/sources/send-data/promtail/_index.md
+18-20
Original file line number
Diff line number
Diff line change
@@ -8,14 +8,16 @@ weight: 300
8
8
---
9
9
# Promtail agent
10
10
11
+
{{< admonition type="caution" >}}
12
+
Promtail is now deprecated and will enter into Long-Term Support (LTS) beginning Feb. 13, 2025. This means that Promtail will no longer receive any new feature updates, but it will receive critical bug fixes and security fixes. Commercial support will end after the LTS phase, which we anticipate will extend for about 12 months until February 28, 2026. End-of-Life (EOL) phase for Promtail will begin once LTS ends. Promtail is expected to reach EOL on March 2, 2026, afterwards no future support or updates will be provided. All future feature development will occur in Grafana Alloy.
13
+
14
+
If you are currently using Promtail, you should plan your [migration to Alloy](https://grafana.com/docs/loki/<LOKI_VERSION>/setup/migrate/migrate-to-alloy/). The Alloy migration documentation includes a migration tool for converting your Promtail configuration to an Alloy configuration with a single command.
15
+
{{< /admonition >}}
16
+
11
17
Promtail is an agent which ships the contents of local logs to a private Grafana Loki
12
18
instance or [Grafana Cloud](/oss/loki). It is usually
13
19
deployed to every machine that runs applications which need to be monitored.
14
20
15
-
{{< admonition type="note" >}}
16
-
Promtail is feature complete. All future feature development will occur in Grafana Alloy.
17
-
{{< /admonition >}}
18
-
19
21
It primarily:
20
22
21
23
- Discovers targets
@@ -74,12 +76,13 @@ scrape_configs:
74
76
```
75
77
76
78
Important details are:
77
-
* It relies on the `\n` character to separate the data into different log lines.
78
-
* The max expected log line is 2MB within the compressed file.
79
-
* The data is decompressed in blocks of 4096 bytes. i.e: it first fetches a block of 4096 bytes
79
+
80
+
- It relies on the `\n` character to separate the data into different log lines.
81
+
- The max expected log line is 2MB within the compressed file.
82
+
- The data is decompressed in blocks of 4096 bytes. i.e: it first fetches a block of 4096 bytes
80
83
from the compressed file and processes it. After processing this block and pushing the data to Loki,
81
84
it fetches the following 4096 bytes, and so on.
82
-
* It supports the following extensions:
85
+
-It supports the following extensions:
83
86
- `.gz`: Data will be decompressed with the native Gunzip Golang pkg (`pkg/compress/gzip`)
84
87
- `.z`: Data will be decompressed with the native Zlib Golang pkg (`pkg/compress/zlib`)
85
88
- `.bz2`: Data will be decompressed with the native Bzip2 Golang pkg (`pkg/compress/bzip2`)
@@ -88,25 +91,20 @@ Important details are:
88
91
compressed file, **the first parsed line will contains metadata together with
* `.zip` extension isn't supported as of now because it doesn't support some of the interfaces
92
-
Promtail requires. We have plans to add support for it in the near future.
93
-
* The decompression is quite CPU intensive and a lot of allocations are expected
94
+
-`.zip`extension isn't supported as of now because it doesn't support some of the interfaces
95
+
Promtail requires.
96
+
-The decompression is quite CPU intensive and a lot of allocations are expected
94
97
to occur, especially depending on the size of the file. You can expect the number
95
98
of garbage collection runs and the CPU usage to skyrocket, but no memory leak is
96
99
expected.
97
-
* Positions are supported. That means that, if you interrupt Promtail after
100
+
-Positions are supported. That means that, if you interrupt Promtail after
98
101
parsing and pushing (for example) 45% of your compressed file data, you can expect Promtail
99
102
to resume work from the last scraped line and process the rest of the remaining 55%.
100
-
* Since decompression and pushing can be very fast, depending on the size
103
+
-Since decompression and pushing can be very fast, depending on the size
101
104
of your compressed file Loki will rate-limit your ingestion. In that case you
102
105
might configure Promtail's [`limits` stage](https://grafana.com/docs/loki/<LOKI_VERSION>/send-data/promtail/configuration/#limits_config) to slow the pace or increase [ingestion limits](https://grafana.com/docs/loki/<LOKI_VERSION>/configure/#limits_config) on Loki.
103
-
104
-
* Log rotations on compressed files **are not supported as of now** (log rotation is fully supported for normal files), mostly because it requires us modifying Promtail to
105
-
rely on file inodes instead of file names. If you'd like to see support for it, create a new
106
-
issue on Github asking for it and explaining your use case.
107
-
* If you compress a file under a folder being scraped, Promtail might try to ingest your file before you finish compressing it. To avoid it, pick a `initial_delay` that is enough to avoid it.
108
-
* If you would like to see support for a compression protocol that isn't listed here, create a new issue on Github asking for it and explaining your use case.
109
-
106
+
- Log rotations on compressed files are not supported (log rotation is fully supported for normal files), mostly because it requires us modifying Promtail to rely on file inodes instead of file names.
107
+
- If you compress a file under a folder being scraped, Promtail might try to ingest your file before you finish compressing it. To avoid it, pick a `initial_delay` that is enough to avoid it.
Sending logs from cloud services to Grafana Loki is a little different depending on the AWS service you are using. The following tutorials walk you through configuring cloud services to send logs to Loki.
Copy file name to clipboardexpand all lines: docs/sources/send-data/promtail/cloud/ec2/_index.md
+2
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,8 @@ weight: 100
11
11
12
12
In this tutorial we're going to setup [Promtail]({{< relref "../../../../send-data/promtail" >}}) on an AWS EC2 instance and configure it to sends all its logs to a Grafana Loki instance.
Copy file name to clipboardexpand all lines: docs/sources/send-data/promtail/cloud/ecs/_index.md
+2
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,8 @@ weight: 100
11
11
12
12
[ECS][ECS] is the fully managed container orchestration service by Amazon. Combined with [Fargate][Fargate] you can run your container workload without the need to provision your own compute resources. In this tutorial we will see how you can leverage [Firelens][Firelens] an AWS log router to forward all your logs and your workload metadata to a Grafana Loki instance.
Copy file name to clipboardexpand all lines: docs/sources/send-data/promtail/cloud/eks/_index.md
+2
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,8 @@ weight: 100
11
11
12
12
In this tutorial we'll see how to set up Promtail on [EKS][eks]. Amazon Elastic Kubernetes Service (Amazon [EKS][eks]) is a fully managed Kubernetes service, using Promtail we'll get full visibility into our cluster logs. We'll start by forwarding pods logs then nodes services and finally Kubernetes events.
Copy file name to clipboardexpand all lines: docs/sources/send-data/promtail/cloud/gcp/_index.md
+2
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,8 @@ weight:
11
11
12
12
This document explains how one can setup Google Cloud Platform to forward its cloud resource logs from a particular GCP project into Google Pubsub topic so that is available for Promtail to consume.
This document assumes, that reader have `gcloud` installed and have the required permissions (as mentioned in [Roles and Permission](#roles-and-permission) section).
The `geoip` stage is a parsing stage that reads an ip address and populates the labelset with geoip fields. [Maxmind's GeoIP2 database](https://www.maxmind.com/en/home) is used for the lookup.
The `multiline` stage merges multiple lines into a multiline block before passing it on to the next stage in the pipeline.
13
15
14
16
A new block is identified by the `firstline` regular expression. Any line that does *not* match the expression is considered to be part of the block of the previous match.
The `pack` stage is a transform stage which lets you embed extracted values and labels into the log line by packing the log line and labels inside a JSON object.
13
15
14
16
For example, if you wanted to remove the labels `container` and `pod` but still wanted to keep their values you could use this stage to create the following output:
0 commit comments