Skip to content

Commit f2d60cb

Browse files
authored
chore(docs): Convert a few more configs to YAML (#18632)
chore(docs): Covnert a few more configs to YAML
1 parent f01354e commit f2d60cb

File tree

3 files changed

+93
-76
lines changed

3 files changed

+93
-76
lines changed

website/content/en/blog/vector-remap-language.md

+27-22
Original file line numberDiff line numberDiff line change
@@ -110,29 +110,34 @@ Parsing this log, without VRL, looks like this:
110110
{{< tabs default="Vector" >}}
111111
{{< tab title="Vector" >}}
112112

113-
```toml title="vector.toml"
113+
```yaml title="vector.yaml"
114114
# ... sources ...
115115

116-
# Parse the internal Syslog log
117-
[transforms.parse_syslog]
118-
type = "regex_parser"
119-
inputs = ["parse_docker"]
120-
patterns = ['^(?P<host>\S+) (?P<client>\S+) (?P<user>\S+) \[(?<timestamp>[\w:/]+\s[+\-]\d{4})\] "(?<method>\S+) (?<resource>.+?) (?<protocol>\S+)" (?<status>\d{3}) (?<bytes_out>\S+)$']
121-
field = "log"
122-
123-
# Remove Docker time and log fields since they are duplicative
124-
[transform.remove_log]
125-
type = "remove_fields"
126-
inputs = ["parse_syslog"]
127-
fields = ["time", "log"]
128-
129-
# Coerce parsed fields
130-
[transforms.coerce_fields]
131-
type = "coercer"
132-
inputs = ["remove_log"]
133-
types.timestamp = "timestamp"
134-
types.status = "int"
135-
types.bytes_out = "int"
116+
transforms:
117+
parse_syslog:
118+
type: regex_parser
119+
inputs:
120+
- parse_docker
121+
patterns:
122+
- '^(?P<host>\S+) (?P<client>\S+) (?P<user>\S+) \[(?<timestamp>[\w:/]+\s[+\-]\d{4})\] "(?<method>\S+) (?<resource>.+?) (?<protocol>\S+)" (?<status>\d{3}) (?<bytes_out>\S+)$'
123+
field: log
124+
125+
remove_log:
126+
type: remove_fields
127+
inputs:
128+
- parse_syslog
129+
fields:
130+
- time
131+
- log
132+
133+
coerce_fields:
134+
type: coercer
135+
inputs:
136+
- remove_log
137+
types:
138+
timestamp: timestamp
139+
status: int
140+
bytes_out: int
136141

137142
# ... sinks ...
138143
```
@@ -224,7 +229,7 @@ transformation.
224229

225230
Instead of conflating these concerns, like Logstash, Fluentd, and others, Vector
226231
separates them, allowing you to choose your preferred configuration language
227-
(TOML, YAML, or JSON) while offering a purpose-built language for data
232+
(YAML, TOML or JSON) while offering a purpose-built language for data
228233
transformation (VRL). But is VRL really necessary? Couldn't you leverage Lua,
229234
JavaScript, or any other existing language?
230235

website/content/en/guides/level-up/transformation.md

+55-46
Original file line numberDiff line numberDiff line change
@@ -56,37 +56,44 @@ create a simple topology consisting of three components:
5656

5757
This configuration defines that topology:
5858

59-
```toml title="vector.toml"
60-
[sources.logs]
61-
type = "demo_logs"
62-
format = "syslog"
63-
interval = 0.1
64-
65-
[transforms.modify]
66-
type = "remap"
67-
inputs = ["logs"]
68-
source = '''
69-
# Parse Syslog input. The "!" means that the script should abort on error.
70-
. = parse_syslog!(.message)
71-
'''
72-
73-
[sinks.out]
74-
type = "console"
75-
inputs = ["modify"]
76-
encoding.codec = "json"
59+
```yaml title="vector.yaml"
60+
sources:
61+
logs:
62+
type: demo_logs
63+
format: syslog
64+
interval: 0.1
65+
66+
transforms:
67+
modify:
68+
type: remap
69+
inputs:
70+
- logs
71+
source: |
72+
# Parse Syslog input. The "!" means that the script should abort on error.
73+
. = parse_syslog!(.message)
74+
75+
sinks:
76+
out:
77+
type: console
78+
inputs:
79+
- modify
80+
encoding:
81+
codec: json
7782
```
7883
7984
{{< info >}}
80-
Although we're using [TOML][urls.toml] for the configuration here, Vector also
81-
supports JSON and YAML.
85+
Although we're using [YAML][urls.yaml] for the configuration here, Vector also
86+
supports [TOML][urls.toml] and [JSON][urls.json].
8287
8388
[urls.toml]: https://github.com/toml-lang/toml
89+
[urls.yaml]: https://yaml.org
90+
[urls.json]: https://www.json.org/json-en.html
8491
{{< /info >}}
8592
8693
To start Vector using this topology:
8794
8895
```bash
89-
vector --config-toml /etc/vector/vector.toml
96+
vector --config /etc/vector/vector.yaml
9097
```
9198

9299
You should see lines like this emitted via stdout (formatted for readability
@@ -109,31 +116,33 @@ So far, we've gotten Vector to *parse* the Syslog data but we're not yet
109116
*modifying* that data. So let's update the `source` script of our `remap`
110117
transform to make some ad hoc transformations:
111118

112-
```toml
113-
[transforms.modify]
114-
type = "remap"
115-
inputs = ["logs"]
116-
source = '''
117-
. = parse_syslog!(.message)
118-
119-
# Convert the timestamp to a Unix timestamp, aborting on error
120-
.timestamp = to_unix_timestamp!(.timestamp)
121-
122-
# Remove the "facility" and "procid" fields
123-
del(.facility); del(.procid)
124-
125-
# Replace the "msgid" field with a unique ID
126-
.msgid = uuid_v4()
127-
128-
# If the log message contains the phrase "Great Scott!", set the new field
129-
# "critical" to true, otherwise set it to false. If the "contains" function
130-
# errors, log the error (instead of aborting the script, as above).
131-
if (is_critical, err = contains(.message, "Great Scott!"); err != null) {
132-
log(err, level: "error")
133-
}
134-
135-
.critical = is_critical
136-
'''
119+
```yaml
120+
transforms:
121+
modify:
122+
type: remap
123+
inputs:
124+
- logs
125+
source: |
126+
. = parse_syslog!(.message)
127+
128+
# Convert the timestamp to a Unix timestamp, aborting on error
129+
.timestamp = to_unix_timestamp!(.timestamp)
130+
131+
# Remove the "facility" and "procid" fields
132+
del(.facility)
133+
del(.procid)
134+
135+
# Replace the "msgid" field with a unique ID
136+
.msgid = uuid_v4()
137+
138+
# If the log message contains the phrase "Great Scott!", set the new field
139+
# "critical" to true, otherwise set it to false. If the "contains" function
140+
# errors, log the error (instead of aborting the script, as above).
141+
if (is_critical, err = contains(.message, "Great Scott!"); err != null) {
142+
log(err, level: "error")
143+
}
144+
145+
.critical = is_critical
137146
```
138147
139148
A few things to notice about this script:

website/cue/reference/components/sinks.cue

+11-8
Original file line numberDiff line numberDiff line change
@@ -601,10 +601,11 @@ components: sinks: [Name=string]: {
601601
If Adaptive Request Concurrency is not for you, you can manually set static concurrency
602602
limits by specifying an integer for `request.concurrency`:
603603
604-
```toml title="vector.toml"
605-
[sinks.my-sink]
606-
request.concurrency = 10
607-
```
604+
```yaml title="vector.yaml"
605+
sinks:
606+
my-sink:
607+
request:
608+
concurrency: 10
608609
"""
609610
},
610611
{
@@ -614,10 +615,12 @@ components: sinks: [Name=string]: {
614615
throughput via the `request.rate_limit_duration_secs` and `request.rate_limit_num`
615616
options.
616617
617-
```toml title="vector.toml"
618-
[sinks.my-sink]
619-
request.rate_limit_duration_secs = 1
620-
request.rate_limit_num = 10
618+
```yaml title="vector.yaml"
619+
sinks:
620+
my-sink:
621+
request:
622+
rate_limit_duration_secs: 1
623+
rate_limit_num: 10
621624
```
622625
623626
These will apply to both `adaptive` and fixed `request.concurrency` values.

0 commit comments

Comments
 (0)