From 16833cd8b841e87d1a73b9a8c8f9c5c778090376 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Thu, 25 Jan 2018 16:59:25 +0100 Subject: [PATCH 01/24] Use specific beat name's instead of 'the Beat' --- libbeat/docs/command-reference.asciidoc | 2 +- libbeat/docs/loggingconfig.asciidoc | 13 ++++++------- libbeat/docs/outputconfig.asciidoc | 14 ++++++++++---- libbeat/docs/processors-using.asciidoc | 2 +- libbeat/docs/repositories.asciidoc | 2 +- libbeat/docs/shared-env-vars.asciidoc | 4 ++-- libbeat/docs/shared-kibana-config.asciidoc | 2 +- libbeat/docs/shared-path-config.asciidoc | 12 +++++++----- libbeat/docs/template-config.asciidoc | 4 ++-- 9 files changed, 31 insertions(+), 24 deletions(-) diff --git a/libbeat/docs/command-reference.asciidoc b/libbeat/docs/command-reference.asciidoc index a8e4cda293d..4dfc12f483b 100644 --- a/libbeat/docs/command-reference.asciidoc +++ b/libbeat/docs/command-reference.asciidoc @@ -31,7 +31,7 @@ Command reference ++++ -{beatname_uc} provides a command-line interface for running the Beat and +{beatname_uc} provides a command-line interface for starting {beatname_uc} and performing common tasks, like testing configuration files and loading dashboards. The command-line also supports <> for controlling global behaviors. diff --git a/libbeat/docs/loggingconfig.asciidoc b/libbeat/docs/loggingconfig.asciidoc index d80355f85e5..94536f34c38 100644 --- a/libbeat/docs/loggingconfig.asciidoc +++ b/libbeat/docs/loggingconfig.asciidoc @@ -14,7 +14,7 @@ == Configure logging The `logging` section of the +{beatname_lc}.yml+ config file contains options -for configuring the Beats logging output. The logging system can write logs to +for configuring the logging output. The logging system can write logs to the syslog or rotate log files. If logging is not explicitly configured the file output is used. @@ -67,7 +67,7 @@ Minimum log level. One of `debug`, `info`, `warning`, or `error`. The default log level is `info`. `debug`:: Logs debug messages, including a detailed printout of all events -flushed by the Beat. Also logs informational messages, warnings, errors, and +flushed. Also logs informational messages, warnings, errors, and critical errors. When the log level is `debug`, you can specify a list of <> to display debug messages for specific components. If no selectors are specified, the `*` selector is used to display debug messages @@ -84,9 +84,9 @@ published. Also logs any warnings, errors, or critical errors. [[selectors]] ==== `logging.selectors` -The list of debugging-only selector tags used by different Beats components. Use `*` -to enable debug output for all components. For example add `publish` to display -all the debug messages related to event publishing. When starting the Beat, +The list of debugging-only selector tags used by different {beatname_uc} components. +Use `*` to enable debug output for all components. For example add `publish` to display +all the debug messages related to event publishing. When starting {beatname_lc}, selectors can be overwritten using the `-d` command line option (`-d` also sets the debug log level). @@ -123,8 +123,7 @@ the <> section for details. [float] ==== `logging.files.name` -The name of the file that logs are written to. By default, the name of the Beat -is used. +The name of the file that logs are written to. The default is '{beatname_lc}'. [float] ==== `logging.files.rotateeverybytes` diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index 04c7ba83fa8..67b98ed4dce 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -20,11 +20,15 @@ output may be defined. The following topics describe how to configure each supported output: * <> + +ifndef::only-elasticsearch[] * <> * <> * <> * <> * <> +endif::[] + [[elasticsearch-output]] === Configure the Elasticsearch output @@ -33,7 +37,7 @@ The following topics describe how to configure each supported output: Elasticsearch ++++ -When you specify Elasticsearch for the output, the Beat sends the transactions directly to Elasticsearch by using the Elasticsearch HTTP API. +When you specify Elasticsearch for the output, {beatname_uc} sends the transactions directly to Elasticsearch by using the Elasticsearch HTTP API. Example configuration: @@ -282,9 +286,8 @@ The default is 3. The maximum number of events to bulk in a single Elasticsearch bulk API index request. The default is 50. -If the Beat sends single events, the events are collected into batches. If the Beat publishes -a large batch of events (larger than the value specified by `bulk_max_size`), the batch is -split. +Events can be collected into batches. {beatname_uc} will split batches larger than `bulk_max_size` +into multiple batches. Specifying a larger batch size can improve performance by lowering the overhead of sending events. However big batch sizes can also increase processing times, which might result in @@ -307,6 +310,7 @@ Elasticsearch. See <> for more information. +ifndef::only-elasticsearch[] [[logstash-output]] === Configure the Logstash output @@ -1162,3 +1166,5 @@ When specified, the `cloud.auth` overwrites the `output.elasticsearch.username` `output.elasticsearch.password` settings. Because the Kibana settings inherit the username and password from the Elasticsearch output, this can also be used to set the `setup.kibana.username` and `setup.kibana.password` options. + +endif::[] diff --git a/libbeat/docs/processors-using.asciidoc b/libbeat/docs/processors-using.asciidoc index c8b6405905e..241663c436e 100644 --- a/libbeat/docs/processors-using.asciidoc +++ b/libbeat/docs/processors-using.asciidoc @@ -543,7 +543,7 @@ default. For example, FileBeat enables the `container` indexer, which indexes pod metadata based on all container IDs, and a `logs_path` matcher, which takes the `source` field, extracts the container ID, and uses it to retrieve metadata. -The configuration below enables the processor when the Beat is run as a pod in +The configuration below enables the processor when {beatname_lc} is run as a pod in Kubernetes. [source,yaml] diff --git a/libbeat/docs/repositories.asciidoc b/libbeat/docs/repositories.asciidoc index 7221a5b4b37..64e9058bd4a 100644 --- a/libbeat/docs/repositories.asciidoc +++ b/libbeat/docs/repositories.asciidoc @@ -88,7 +88,7 @@ install {beatname_uc} by running: sudo apt-get update && sudo apt-get install {beatname_pkg} -------------------------------------------------- -. To configure the Beat to start automatically during boot, run: +. To configure {beatname_uc} to start automatically during boot, run: + ["source","sh",subs="attributes"] -------------------------------------------------- diff --git a/libbeat/docs/shared-env-vars.asciidoc b/libbeat/docs/shared-env-vars.asciidoc index d04b7b6df03..496f6aa480c 100644 --- a/libbeat/docs/shared-env-vars.asciidoc +++ b/libbeat/docs/shared-env-vars.asciidoc @@ -49,7 +49,7 @@ If you need to use a literal `${` in your configuration file then you can write `$${` to escape the expansion. After changing the value of an environment variable, you need to restart -the Beat to pick up the new value. +{beatname_uc} to pick up the new value. [NOTE] ================================== @@ -104,7 +104,7 @@ output.elasticsearch: hosts: '${ES_HOSTS}' ------------------------------------------------------------------------------- -When the Beat loads the config file, it resolves the environment variable and +When {beatname_uc} loads the config file, it resolves the environment variable and replaces it with the specified list before reading the `hosts` setting. NOTE: Do not use double-quotes (`"`) to wrap regular expressions, or the backslash (`\`) will be interpreted as an escape character. diff --git a/libbeat/docs/shared-kibana-config.asciidoc b/libbeat/docs/shared-kibana-config.asciidoc index 41e0586894c..f9a0a5bc78f 100644 --- a/libbeat/docs/shared-kibana-config.asciidoc +++ b/libbeat/docs/shared-kibana-config.asciidoc @@ -89,7 +89,7 @@ under a custom prefix. ==== `setup.kibana.ssl.enabled` Enables {beatname_uc} to use SSL settings when connecting to Kibana via HTTPS. -If you configure the Beat to connect over HTTPS, this setting defaults to +If you configure {beatname_uc} to connect over HTTPS, this setting defaults to `true` and {beatname_uc} uses the default SSL settings. Example configuration: diff --git a/libbeat/docs/shared-path-config.asciidoc b/libbeat/docs/shared-path-config.asciidoc index f08d2810c68..073f9015fc5 100644 --- a/libbeat/docs/shared-path-config.asciidoc +++ b/libbeat/docs/shared-path-config.asciidoc @@ -14,10 +14,12 @@ == Set up project paths The `path` section of the +{beatname_lc}.yml+ config file contains configuration -options that define where the Beat looks for its files. For example, all Beats -look for the Elasticsearch template file in the configuration path, Filebeat and -Winlogbeat look for their registry files in the data path, and all Beats write -their log files in the logs path. +options that define where {beatname_lc} looks for its files. For example, {beatname_uc} +looks for the Elasticsearch template file in the configuration path and writes +log files in the logs path. +ifeval::["{beatname_lc}"=="filebeat" or "{beatname_lc}"=="winlogbeat"] +Filebeat and Winlogbeat look for their registry files in the data path. +endif::[] Please see the <> section for more details. @@ -87,7 +89,7 @@ path.data: /var/lib/beats [float] ==== `logs` -The logs path for a {beatname_uc} installation. This is the default location for the Beat's +The logs path for a {beatname_uc} installation. This is the default location for {beatname_uc}'s log files. If not set by a CLI flag or in the configuration file, the default for the logs path is a `logs` subdirectory inside the home path. diff --git a/libbeat/docs/template-config.asciidoc b/libbeat/docs/template-config.asciidoc index 85b79fc3746..9447c437b5e 100644 --- a/libbeat/docs/template-config.asciidoc +++ b/libbeat/docs/template-config.asciidoc @@ -19,7 +19,7 @@ existing one. you must <>. *`setup.template.name`*:: The name of the template. The default is -+{beatname_lc}+. The Beat version is always appended to the given ++{beatname_lc}+. The {beatname_uc} version is always appended to the given name, so the final name is +{beatname_lc}-%\{[beat.version]\}+. // Maintainers: a backslash character is required to escape curly braces and @@ -29,7 +29,7 @@ name, so the final name is +{beatname_lc}-%\{[beat.version]\}+. // the example as expected. *`setup.template.pattern`*:: The template pattern to apply to the default index -settings. The default pattern is +{beatname_lc}-\*+. The Beat version is always +settings. The default pattern is +{beatname_lc}-\*+. The {beatname_uc} version is always included in the pattern, so the final pattern is +{beatname_lc}-%\{[beat.version]\}-*+. The wildcard character `-*` is used to match all daily indices. From 54cc9f49e0c4e33037a6b9fcbf634be573bc7eb6 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Thu, 25 Jan 2018 19:25:15 +0100 Subject: [PATCH 02/24] Dont show Logstash info unless it's supported. --- libbeat/docs/shared-ssl-config.asciidoc | 4 +++- libbeat/docs/template-config.asciidoc | 4 ++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/libbeat/docs/shared-ssl-config.asciidoc b/libbeat/docs/shared-ssl-config.asciidoc index 2aa6f3b8110..c1111d83d6c 100644 --- a/libbeat/docs/shared-ssl-config.asciidoc +++ b/libbeat/docs/shared-ssl-config.asciidoc @@ -1,5 +1,5 @@ [[configuration-ssl]] -== Specify SSL settings +== SSL settings for outputs You can specify SSL options for any <> that supports SSL. You can also specify SSL options when you @@ -15,7 +15,9 @@ output.elasticsearch.ssl.certificate: "/etc/pki/client/cert.pem" output.elasticsearch.ssl.key: "/etc/pki/client/cert.key" ---- +ifndef::only-elasticsearch[] Also see <>. +endif::[] Example Kibana endpoint config with SSL enabled: diff --git a/libbeat/docs/template-config.asciidoc b/libbeat/docs/template-config.asciidoc index 9447c437b5e..fbe079918f3 100644 --- a/libbeat/docs/template-config.asciidoc +++ b/libbeat/docs/template-config.asciidoc @@ -8,10 +8,14 @@ mappings in Elasticsearch. If template loading is enabled (the default), {beatname_uc} loads the index template automatically after successfully connecting to Elasticsearch. +ifndef::only-elasticsearch[] + NOTE: A connection to Elasticsearch is required to load the index template. If the output is Logstash, you must <>. +endif::[] + You can adjust the following settings to load your own template or overwrite an existing one. From 0539799996b88dbfb05336669becbb1f0260aa75 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Thu, 25 Jan 2018 19:50:27 +0100 Subject: [PATCH 03/24] Dont show Logstash info if not supported. --- libbeat/docs/shared-template-load.asciidoc | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/libbeat/docs/shared-template-load.asciidoc b/libbeat/docs/shared-template-load.asciidoc index 20fb21e4922..9ca7d0e0217 100644 --- a/libbeat/docs/shared-template-load.asciidoc +++ b/libbeat/docs/shared-template-load.asciidoc @@ -31,14 +31,23 @@ configuring template loading options in the {beatname_uc} configuration file. You can also set options to change the name of the index and index template. +ifndef::only-elasticsearch[] NOTE: A connection to Elasticsearch is required to load the index template. If the output is Logstash, you must <>. +endif::[] For more information, see: -* <> +ifdef::only-elasticsearch[] +* <> +* <> +endif::[] + +ifndef::only-elasticsearch[] +* <> * <> - required for Logstash output +endif::[] [[load-template-auto]] ==== Configure template loading From 74f2e49560ac9e40908b805188a35218a533a8db Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Thu, 25 Jan 2018 20:38:01 +0100 Subject: [PATCH 04/24] Make the index pattern decoupled from the beat name --- auditbeat/docs/index.asciidoc | 1 + filebeat/docs/index.asciidoc | 1 + heartbeat/docs/index.asciidoc | 2 ++ libbeat/docs/dashboards.asciidoc | 5 ++--- libbeat/docs/index.asciidoc | 2 +- metricbeat/docs/index.asciidoc | 1 + packetbeat/docs/index.asciidoc | 1 + winlogbeat/docs/index.asciidoc | 1 + 8 files changed, 10 insertions(+), 4 deletions(-) diff --git a/auditbeat/docs/index.asciidoc b/auditbeat/docs/index.asciidoc index 73e9d0836b4..b40cb0452d8 100644 --- a/auditbeat/docs/index.asciidoc +++ b/auditbeat/docs/index.asciidoc @@ -10,6 +10,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :beatname_pkg: {beatname_lc} :github_repo_name: beats :discuss_forum: beats/{beatname_lc} +:beat_default_index_prefix: {beatname_lc} include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/filebeat/docs/index.asciidoc b/filebeat/docs/index.asciidoc index 4d7bc88b56a..f6adb790f7f 100644 --- a/filebeat/docs/index.asciidoc +++ b/filebeat/docs/index.asciidoc @@ -10,6 +10,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :beatname_pkg: {beatname_lc} :github_repo_name: beats :discuss_forum: beats/{beatname_lc} +:beat_default_index_prefix: {beatname_lc} include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/heartbeat/docs/index.asciidoc b/heartbeat/docs/index.asciidoc index 00931d0f44d..826aa90a610 100644 --- a/heartbeat/docs/index.asciidoc +++ b/heartbeat/docs/index.asciidoc @@ -10,6 +10,8 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :beatname_pkg: heartbeat-elastic :github_repo_name: beats :discuss_forum: beats/{beatname_lc} +:beat_default_index_prefix: {beatname_lc} + include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/libbeat/docs/dashboards.asciidoc b/libbeat/docs/dashboards.asciidoc index 5547343f821..f0d62f5308d 100644 --- a/libbeat/docs/dashboards.asciidoc +++ b/libbeat/docs/dashboards.asciidoc @@ -9,16 +9,15 @@ //// include::../../libbeat/docs/dashboards.asciidoc[] ////////////////////////////////////////////////////////////////////////// - {beatname_uc} comes packaged with example Kibana dashboards, visualizations, and searches for visualizing {beatname_uc} data in Kibana. Before you can use -the dashboards, you need to create the index pattern, +{beatname_lc}-*+, and +the dashboards, you need to create the index pattern, +{beat_default_index_prefix}-*+, and load the dashboards into Kibana. To do this, you can either run the `setup` command (as described here) or <> in the +{beatname_lc}.yml+ config file. -NOTE: Starting with Beats 6.0.0, the dashboards are loaded via the Kibana API. +NOTE: Starting with {beatname_uc} 6.0.0, the dashboards are loaded via the Kibana API. This requires a Kibana endpoint configuration. You should have configured the endpoint earlier when you <<{beatname_lc}-configuration,configured {beatname_uc}>>. If you didn't, diff --git a/libbeat/docs/index.asciidoc b/libbeat/docs/index.asciidoc index e3e0cda0fd6..c99c00f291e 100644 --- a/libbeat/docs/index.asciidoc +++ b/libbeat/docs/index.asciidoc @@ -10,7 +10,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :beatname_pkg: {beatname_lc} :github_repo_name: beats :discuss_forum: beats/{beatname_lc} - +:beat_default_index_prefix: {beatname_lc} include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/metricbeat/docs/index.asciidoc b/metricbeat/docs/index.asciidoc index 19a8e59092b..18544002f59 100644 --- a/metricbeat/docs/index.asciidoc +++ b/metricbeat/docs/index.asciidoc @@ -10,6 +10,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :beatname_pkg: {beatname_lc} :github_repo_name: beats :discuss_forum: beats/{beatname_lc} +:beat_default_index_prefix: {beatname_lc} include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/packetbeat/docs/index.asciidoc b/packetbeat/docs/index.asciidoc index f49d086042d..efa54886f12 100644 --- a/packetbeat/docs/index.asciidoc +++ b/packetbeat/docs/index.asciidoc @@ -10,6 +10,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :beatname_pkg: {beatname_lc} :github_repo_name: beats :discuss_forum: beats/{beatname_lc} +:beat_default_index_prefix: {beatname_lc} include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/winlogbeat/docs/index.asciidoc b/winlogbeat/docs/index.asciidoc index e6215096ea6..62a33e32397 100644 --- a/winlogbeat/docs/index.asciidoc +++ b/winlogbeat/docs/index.asciidoc @@ -10,6 +10,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :beatname_pkg: {beatname_lc} :github_repo_name: beats :discuss_forum: beats/{beatname_lc} +:beat_default_index_prefix: {beatname_lc} include::../../libbeat/docs/shared-beats-attributes.asciidoc[] From 0b16e740395ec521b3e829e8234ca7c7876cb2e3 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Thu, 25 Jan 2018 20:38:18 +0100 Subject: [PATCH 05/24] Make it possible to skip the pipeline docs in output.elasticsearch --- libbeat/docs/outputconfig.asciidoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index 67b98ed4dce..ba23898c7b4 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -226,6 +226,7 @@ output.elasticsearch: message: "ERR" ------------------------------------------------------------------------------ +ifndef::no-pipeline[] ===== `pipeline` A format string value that specifies the ingest node pipeline to write events to. @@ -270,6 +271,7 @@ output.elasticsearch: when.equals: fields.type: "normal" ------------------------------------------------------------------------------ +endif::[] ===== `max_retries` From 4396b35b04809efc6dd688db894550f30ae6b1b4 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Thu, 25 Jan 2018 23:02:34 +0100 Subject: [PATCH 06/24] Use the specific beat name. --- libbeat/docs/dashboardsconfig.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libbeat/docs/dashboardsconfig.asciidoc b/libbeat/docs/dashboardsconfig.asciidoc index 3dd255009a5..6670dad1ce6 100644 --- a/libbeat/docs/dashboardsconfig.asciidoc +++ b/libbeat/docs/dashboardsconfig.asciidoc @@ -20,7 +20,7 @@ To load the dashboards, you can either enable dashboard loading in the run the `setup` command. Dashboard loading is disabled by default. When dashboard loading is enabled, {beatname_uc} uses the Kibana API to load the -sample dashboards. Dashboard loading is only attempted at Beat startup. +sample dashboards. Dashboard loading is only attempted when {beatname_uc} startups up. If Kibana is not available at startup, {beatname_uc} will stop with an error. To enable dashboard loading, add the following setting to the config file: From 5745abf576b1ccc6e6c1578f1a2df1cf8a67b82c Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Fri, 26 Jan 2018 13:53:34 +0100 Subject: [PATCH 07/24] Fix typo as suggested by @dedemorton. --- libbeat/docs/dashboardsconfig.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libbeat/docs/dashboardsconfig.asciidoc b/libbeat/docs/dashboardsconfig.asciidoc index 6670dad1ce6..dd5e255d22f 100644 --- a/libbeat/docs/dashboardsconfig.asciidoc +++ b/libbeat/docs/dashboardsconfig.asciidoc @@ -20,7 +20,7 @@ To load the dashboards, you can either enable dashboard loading in the run the `setup` command. Dashboard loading is disabled by default. When dashboard loading is enabled, {beatname_uc} uses the Kibana API to load the -sample dashboards. Dashboard loading is only attempted when {beatname_uc} startups up. +sample dashboards. Dashboard loading is only attempted when {beatname_uc} starts up. If Kibana is not available at startup, {beatname_uc} will stop with an error. To enable dashboard loading, add the following setting to the config file: From 97e36484e5038fd37b06028ec6cd8d42c0465077 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Fri, 26 Jan 2018 15:29:38 +0100 Subject: [PATCH 08/24] More beat_default_index_prefix. --- libbeat/docs/template-config.asciidoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libbeat/docs/template-config.asciidoc b/libbeat/docs/template-config.asciidoc index fbe079918f3..6b82e1e0856 100644 --- a/libbeat/docs/template-config.asciidoc +++ b/libbeat/docs/template-config.asciidoc @@ -23,7 +23,7 @@ existing one. you must <>. *`setup.template.name`*:: The name of the template. The default is -+{beatname_lc}+. The {beatname_uc} version is always appended to the given ++{beatname_lc}-*+. The {beatname_uc} version is always appended to the given name, so the final name is +{beatname_lc}-%\{[beat.version]\}+. // Maintainers: a backslash character is required to escape curly braces and @@ -33,9 +33,9 @@ name, so the final name is +{beatname_lc}-%\{[beat.version]\}+. // the example as expected. *`setup.template.pattern`*:: The template pattern to apply to the default index -settings. The default pattern is +{beatname_lc}-\*+. The {beatname_uc} version is always +settings. The default pattern is +{beat_default_index_prefix}-\*+. The {beatname_uc} version is always included in the pattern, so the final pattern is -+{beatname_lc}-%\{[beat.version]\}-*+. The wildcard character `-*` is used to ++{beat_default_index_prefix}-%\{[beat.version]\}-*+. The wildcard character `-*` is used to match all daily indices. + Example: @@ -43,7 +43,7 @@ Example: ["source","yaml",subs="attributes"] ---------------------------------------------------------------------- setup.template.name: "{beatname_lc}" -setup.template.pattern: "{beatname_lc}-*" +setup.template.pattern: "{beat_default_index_prefix}-*" ---------------------------------------------------------------------- *`setup.template.fields`*:: The path to the YAML file describing the fields. The default is +fields.yml+. If a From 3d41272b5e51e1447cd313fd707a284b26124a86 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Fri, 26 Jan 2018 16:01:28 +0100 Subject: [PATCH 09/24] Use name of beat in shared-kibana-config.asciidoc --- libbeat/docs/shared-kibana-config.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libbeat/docs/shared-kibana-config.asciidoc b/libbeat/docs/shared-kibana-config.asciidoc index f9a0a5bc78f..4525cde26e6 100644 --- a/libbeat/docs/shared-kibana-config.asciidoc +++ b/libbeat/docs/shared-kibana-config.asciidoc @@ -12,7 +12,7 @@ [[setup-kibana-endpoint]] == Set up the Kibana endpoint -Starting with Beats 6.0.0, the Kibana dashboards are loaded into Kibana +Starting with {beatname_uc} 6.0.0, the Kibana dashboards are loaded into Kibana via the Kibana API. This requires a Kibana endpoint configuration. You configure the endpoint in the `setup.kibana` section of the From dd8aef8b87363105708b3902ce3567f907e7578f Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 11:23:37 +0100 Subject: [PATCH 10/24] Add .\ for PS instruction and make it possible to remove logstash mention. --- libbeat/docs/shared-template-load.asciidoc | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/libbeat/docs/shared-template-load.asciidoc b/libbeat/docs/shared-template-load.asciidoc index 9ca7d0e0217..3677a1091a1 100644 --- a/libbeat/docs/shared-template-load.asciidoc +++ b/libbeat/docs/shared-template-load.asciidoc @@ -113,10 +113,13 @@ See <> for the full list of configuration options. ==== Load the template manually To load the template manually, run the <> command. A -connection to Elasticsearch is required. If Logstash output is enabled, you need +connection to Elasticsearch is required. +ifndef::only-elasticsearch[] +If Logstash output is enabled, you need to temporarily disable the Logstash output and enable Elasticsearch by using the `-E` option. The examples here assume that Logstash output is enabled. You can omit the `-E` flags if Elasticsearch output is already enabled. +endif::[] If you are connecting to a secured Elasticsearch cluster, make sure you've configured credentials as described in <<{beatname_lc}-configuration>>. @@ -126,6 +129,14 @@ Elasticsearch, see <>. To load the template, use the appropriate command for your system. +ifndef::only-elasticsearch[] +:disable_logstash: {sp}-E output.logstash.enabled=false +endif::[] + +ifdef::only-elasticsearch[] +:disable_logstash: +endif::[] + ifdef::allplatforms[] ifeval::["{requires-sudo}"=="yes"] @@ -135,17 +146,16 @@ include::./shared-note-sudo.asciidoc[] endif::[] *deb and rpm:* - ["source","sh",subs="attributes"] ---- -{beatname_lc} setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]' +{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---- *mac:* ["source","sh",subs="attributes"] ---- -./{beatname_lc} setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]' +./{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---- @@ -155,7 +165,7 @@ ifeval::["{beatname_lc}"!="auditbeat"] ["source","sh",subs="attributes"] ---------------------------------------------------------------------- -docker run {dockerimage} setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]' +docker run {dockerimage} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---------------------------------------------------------------------- @@ -174,7 +184,7 @@ and run: ["source","sh",subs="attributes,callouts"] ---------------------------------------------------------------------- -PS > {beatname_lc} setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]' +PS > .{backslash}{beatname_lc} setup --template{disable_logstash} -E 'output.elasticsearch.hosts=["localhost:9200"]' ---------------------------------------------------------------------- From fe3fb160b09897e8e721eb4fab2f4523ab2c6529 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 11:37:34 +0100 Subject: [PATCH 11/24] Introduce 'has_ml_jobs' --- auditbeat/docs/index.asciidoc | 1 + filebeat/docs/index.asciidoc | 1 + heartbeat/docs/index.asciidoc | 2 +- libbeat/docs/command-reference.asciidoc | 6 ++++++ libbeat/docs/index.asciidoc | 1 + metricbeat/docs/index.asciidoc | 1 + packetbeat/docs/index.asciidoc | 1 + winlogbeat/docs/index.asciidoc | 1 + 8 files changed, 13 insertions(+), 1 deletion(-) diff --git a/auditbeat/docs/index.asciidoc b/auditbeat/docs/index.asciidoc index b40cb0452d8..cbe41fbd61b 100644 --- a/auditbeat/docs/index.asciidoc +++ b/auditbeat/docs/index.asciidoc @@ -11,6 +11,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :github_repo_name: beats :discuss_forum: beats/{beatname_lc} :beat_default_index_prefix: {beatname_lc} +:has_ml_jobs: yes include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/filebeat/docs/index.asciidoc b/filebeat/docs/index.asciidoc index f6adb790f7f..47f4a65d835 100644 --- a/filebeat/docs/index.asciidoc +++ b/filebeat/docs/index.asciidoc @@ -11,6 +11,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :github_repo_name: beats :discuss_forum: beats/{beatname_lc} :beat_default_index_prefix: {beatname_lc} +:has_ml_jobs: yes include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/heartbeat/docs/index.asciidoc b/heartbeat/docs/index.asciidoc index 826aa90a610..7dd3af34e97 100644 --- a/heartbeat/docs/index.asciidoc +++ b/heartbeat/docs/index.asciidoc @@ -11,7 +11,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :github_repo_name: beats :discuss_forum: beats/{beatname_lc} :beat_default_index_prefix: {beatname_lc} - +:has_ml_jobs: yes include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/libbeat/docs/command-reference.asciidoc b/libbeat/docs/command-reference.asciidoc index 4dfc12f483b..ac5c96df767 100644 --- a/libbeat/docs/command-reference.asciidoc +++ b/libbeat/docs/command-reference.asciidoc @@ -19,7 +19,13 @@ :keystore-command-short-desc: Manages the <> :modules-command-short-desc: Manages configured modules :run-command-short-desc: Runs {beatname_uc}. This command is used by default if you start {beatname_uc} without specifying a command + +ifeval::["{has_ml_jobs}"=="yes"] :setup-command-short-desc: Sets up the initial environment, including the index template, Kibana dashboards (when available), and machine learning jobs (when available) +else::[] +:setup-command-short-desc: Sets up the initial environment, including the index template, Kibana dashboards (when available) +endif::[] + :test-command-short-desc: Tests the configuration :version-command-short-desc: Shows information about the current version diff --git a/libbeat/docs/index.asciidoc b/libbeat/docs/index.asciidoc index c99c00f291e..368e41c43f6 100644 --- a/libbeat/docs/index.asciidoc +++ b/libbeat/docs/index.asciidoc @@ -11,6 +11,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :github_repo_name: beats :discuss_forum: beats/{beatname_lc} :beat_default_index_prefix: {beatname_lc} +:has_ml_jobs: yes include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/metricbeat/docs/index.asciidoc b/metricbeat/docs/index.asciidoc index 18544002f59..4eab69b5070 100644 --- a/metricbeat/docs/index.asciidoc +++ b/metricbeat/docs/index.asciidoc @@ -11,6 +11,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :github_repo_name: beats :discuss_forum: beats/{beatname_lc} :beat_default_index_prefix: {beatname_lc} +:has_ml_jobs: yes include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/packetbeat/docs/index.asciidoc b/packetbeat/docs/index.asciidoc index efa54886f12..df072e50514 100644 --- a/packetbeat/docs/index.asciidoc +++ b/packetbeat/docs/index.asciidoc @@ -11,6 +11,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :github_repo_name: beats :discuss_forum: beats/{beatname_lc} :beat_default_index_prefix: {beatname_lc} +:has_ml_jobs: yes include::../../libbeat/docs/shared-beats-attributes.asciidoc[] diff --git a/winlogbeat/docs/index.asciidoc b/winlogbeat/docs/index.asciidoc index 62a33e32397..83a19e70bd2 100644 --- a/winlogbeat/docs/index.asciidoc +++ b/winlogbeat/docs/index.asciidoc @@ -11,6 +11,7 @@ include::{asciidoc-dir}/../../shared/attributes.asciidoc[] :github_repo_name: beats :discuss_forum: beats/{beatname_lc} :beat_default_index_prefix: {beatname_lc} +:has_ml_jobs: yes include::../../libbeat/docs/shared-beats-attributes.asciidoc[] From 81b40cfa05247d1c48e0086b2c63d5fe61c22690 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 12:33:43 +0100 Subject: [PATCH 12/24] Added `html_docs` to .gitignore --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index fa69db58a98..56e3cccf372 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ /*/logs /*/fields.yml /*/*.template*.json +**/html_docs # Files .DS_Store From 29e1fecfd47b91a8d84a60fae899d4fb217fd7e5 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 12:42:41 +0100 Subject: [PATCH 13/24] Better outputconfig for Elasticsearch only beats. --- libbeat/docs/outputconfig.asciidoc | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index ba23898c7b4..552e227c24b 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -13,15 +13,18 @@ [[configuring-output]] == Configure the output +ifdef::only-elasticsearch[] +You configure {beatname_uc} to write to Elasticsearch by setting options in +the `output.elasticsearch` of the +{beatname_lc}.yml+ config file. +endif::[] + +ifndef::only-elasticsearch[] You configure {beatname_uc} to write to a specific output by setting options in the `output` section of the +{beatname_lc}.yml+ config file. Only a single output may be defined. The following topics describe how to configure each supported output: - * <> - -ifndef::only-elasticsearch[] * <> * <> * <> @@ -29,7 +32,6 @@ ifndef::only-elasticsearch[] * <> endif::[] - [[elasticsearch-output]] === Configure the Elasticsearch output From 0219cffa94ecefc9cb4a7b2c980014e02e85e6d6 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 12:43:33 +0100 Subject: [PATCH 14/24] Less blamy wording and .. removed version note as these docs are only for this version --- libbeat/docs/dashboards.asciidoc | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/libbeat/docs/dashboards.asciidoc b/libbeat/docs/dashboards.asciidoc index f0d62f5308d..3c76c4d91ca 100644 --- a/libbeat/docs/dashboards.asciidoc +++ b/libbeat/docs/dashboards.asciidoc @@ -17,11 +17,8 @@ command (as described here) or <> in the +{beatname_lc}.yml+ config file. -NOTE: Starting with {beatname_uc} 6.0.0, the dashboards are loaded via the Kibana API. -This requires a Kibana endpoint configuration. You should have configured the -endpoint earlier when you -<<{beatname_lc}-configuration,configured {beatname_uc}>>. If you didn't, -configure it now. +This requires a Kibana endpoint configuration. If you didn't already configure +a Kibana endpoint, see <<{beatname_lc}-configuration,configured {beatname_uc}>> Make sure Kibana is running before you perform this step. If you are accessing a secured Kibana instance, make sure you've configured credentials as described in From e3f055db3c76add0265af3f53338e3ddfec0e7af Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 13:09:46 +0100 Subject: [PATCH 15/24] Fix list in output config --- libbeat/docs/outputconfig.asciidoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index 552e227c24b..588881864bc 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -24,12 +24,14 @@ in the `output` section of the +{beatname_lc}.yml+ config file. Only a single output may be defined. The following topics describe how to configure each supported output: + * <> * <> * <> * <> * <> * <> + endif::[] [[elasticsearch-output]] From 5b25d58dc1dca535856a43acb8b59c9452107b82 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 13:10:05 +0100 Subject: [PATCH 16/24] Only talk about Filebeat for filebeat docs --- libbeat/docs/outputconfig.asciidoc | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index 588881864bc..cda804e9bfc 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -281,10 +281,13 @@ endif::[] The number of times to retry publishing an event after a publishing failure. After the specified number of retries, the events are typically dropped. -Some Beats, such as Filebeat, ignore the `max_retries` setting and retry until all -events are published. +ifeval::["{beatname_lc}" == "filebeat"] +Filebeat will ignore the `max_retries` setting and retry until all +events are published. +ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. +endif::[] The default is 3. From 06ff1232a017af3c52d7f9bbc437b198eac5f6ac Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 13:25:02 +0100 Subject: [PATCH 17/24] It's 'warn', not 'warning'. Found by @simitt --- libbeat/docs/loggingconfig.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libbeat/docs/loggingconfig.asciidoc b/libbeat/docs/loggingconfig.asciidoc index 94536f34c38..76c578de8d6 100644 --- a/libbeat/docs/loggingconfig.asciidoc +++ b/libbeat/docs/loggingconfig.asciidoc @@ -63,7 +63,7 @@ errors, there will be no log file in the directory specified for logs. [[level]] ==== `logging.level` -Minimum log level. One of `debug`, `info`, `warning`, or `error`. The default +Minimum log level. One of `debug`, `info`, `warn`, or `error`. The default log level is `info`. `debug`:: Logs debug messages, including a detailed printout of all events @@ -76,7 +76,7 @@ for all components. `info`:: Logs informational messages, including the number of events that are published. Also logs any warnings, errors, or critical errors. -`warning`:: Logs warnings, errors, and critical errors. +`warn`:: Logs warnings, errors, and critical errors. `error`:: Logs errors and critical errors. From 5319f248d8ca3625575684565784e12fa587284f Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 13:35:05 +0100 Subject: [PATCH 18/24] Update the rest of the max_retries sections to only talk about Filebeat for filebeat --- libbeat/docs/outputconfig.asciidoc | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index cda804e9bfc..be2fdb1a558 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -528,10 +528,13 @@ The number of seconds to wait for responses from the Logstash server before timi The number of times to retry publishing an event after a publishing failure. After the specified number of retries, the events are typically dropped. -Some Beats, such as Filebeat, ignore the `max_retries` setting and retry until all -events are published. +ifeval::["{beatname_lc}" == "filebeat"] +Filebeat will ignore the `max_retries` setting and retry until all +events are published. +ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. +endif::[] The default is 3. @@ -719,10 +722,13 @@ brokers, topics, partition, and active leaders to use for publishing. The number of times to retry publishing an event after a publishing failure. After the specified number of retries, the events are typically dropped. -Some Beats, such as Filebeat, ignore the `max_retries` setting and retry until all -events are published. +ifeval::["{beatname_lc}" == "filebeat"] +Filebeat will ignore the `max_retries` setting and retry until all +events are published. +ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. +endif::[] The default is 3. @@ -941,13 +947,17 @@ The Redis connection timeout in seconds. The default is 5 seconds. The number of times to retry publishing an event after a publishing failure. After the specified number of retries, the events are typically dropped. -Some Beats, such as Filebeat, ignore the `max_retries` setting and retry until all -events are published. +ifeval::["{beatname_lc}" == "filebeat"] +Filebeat will ignore the `max_retries` setting and retry until all +events are published. +ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. +endif::[] The default is 3. + ===== `bulk_max_size` The maximum number of events to bulk in a single Redis request or pipeline. The default is 2048. From 58809992d60d363a038cb75c984167511658eb31 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Wed, 31 Jan 2018 13:40:42 +0100 Subject: [PATCH 19/24] Special case for apm-server as it was only introduced in 6.0 --- libbeat/docs/shared-kibana-config.asciidoc | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/libbeat/docs/shared-kibana-config.asciidoc b/libbeat/docs/shared-kibana-config.asciidoc index 4525cde26e6..8b1f79721bb 100644 --- a/libbeat/docs/shared-kibana-config.asciidoc +++ b/libbeat/docs/shared-kibana-config.asciidoc @@ -12,8 +12,16 @@ [[setup-kibana-endpoint]] == Set up the Kibana endpoint +ifeval::["{beatname_lc} == "apm-server"] +The Kibana dashboards are loaded into Kibana via the Kibana API. +This requires a Kibana endpoint configuration. +endif::[] + +ifeval::["{beatname_lc} != "apm-server"] Starting with {beatname_uc} 6.0.0, the Kibana dashboards are loaded into Kibana via the Kibana API. This requires a Kibana endpoint configuration. +endif::[] + You configure the endpoint in the `setup.kibana` section of the +{beatname_lc}.yml+ config file. From 98dd57ad3a0f13324bc9f2f1af226189a2f8e21b Mon Sep 17 00:00:00 2001 From: Gil Raphaelli Date: Fri, 2 Feb 2018 10:14:10 -0500 Subject: [PATCH 20/24] expand beat_default_index_prefix use (#2) --- libbeat/docs/security/basic-auth.asciidoc | 26 +++++++++++----------- libbeat/docs/security/user-access.asciidoc | 20 ++++++++--------- libbeat/docs/shared-template-load.asciidoc | 2 +- 3 files changed, 24 insertions(+), 24 deletions(-) diff --git a/libbeat/docs/security/basic-auth.asciidoc b/libbeat/docs/security/basic-auth.asciidoc index 85fd6decfbf..ece4ce31665 100644 --- a/libbeat/docs/security/basic-auth.asciidoc +++ b/libbeat/docs/security/basic-auth.asciidoc @@ -12,23 +12,23 @@ To configure authentication credentials for {beatname_uc}: `monitor` cluster privileges, and `read`, `write`, and `create_index` privileges for the indices that {beatname_uc} creates. You can create roles from the **Management / Roles** UI in {kib} or through the `role` API. -For example, the following request creates a ++{beatname_lc}_writer++ role: +For example, the following request creates a ++{beat_default_index_prefix}_writer++ role: + ["source","sh",subs="attributes,callouts"] --------------------------------------------------------------- -POST _xpack/security/role/{beatname_lc}_writer +POST _xpack/security/role/{beat_default_index_prefix}_writer { "cluster": ["manage_index_templates", "monitor"], "indices": [ { - "names": [ "{beatname_lc}-*" ], <1> + "names": [ "{beat_default_index_prefix}-*" ], <1> "privileges": ["write","create_index"] } ] } --------------------------------------------------------------- <1> If you use a custom {beatname_uc} index pattern, specify that pattern -instead of the default ++{beatname_lc}-*++ pattern. +instead of the default ++{beat_default_index_prefix}-*++ pattern. . Assign the writer role to the user that {beatname_uc} will use to connect to {es}: @@ -36,15 +36,15 @@ instead of the default ++{beatname_lc}-*++ pattern. .. To authenticate as a native user, create a user for the {beatname_uc} to use internally and assign it the writer role. You can create users from the **Management / Users** UI in {kib} or through the `user` API. For example, the -following request creates a ++{beatname_lc}_internal++ user that has the -++{beatname_lc}_writer++ role: +following request creates a ++{beat_default_index_prefix}_internal++ user that has the +++{beat_default_index_prefix}_writer++ role: + ["source","sh",subs="attributes,callouts"] --------------------------------------------------------------- -POST /_xpack/security/user/{beatname_lc}_internal +POST /_xpack/security/user/{beat_default_index_prefix}_internal { "password" : "x-pack-test-password", - "roles" : [ "{beatname_lc}_writer"], + "roles" : [ "{beat_default_index_prefix}_writer"], "full_name" : "Internal {beatname_uc} User" } --------------------------------------------------------------- @@ -56,7 +56,7 @@ the user by the distinguished name that appears in its certificate. -- ["source","yaml",subs="attributes,callouts"] --------------------------------------------------------------- -{beatname_lc}_writer: +{beat_default_index_prefix}_writer: - "cn=Internal {beatname_uc} User,ou=example,o=com" --------------------------------------------------------------- For more information, see @@ -68,14 +68,14 @@ in the {beatname_uc} configuration file: .. To use basic authentication, configure the `username` and `password` settings. For example, the following {beatname_uc} output configuration -uses the native ++{beatname_lc}_internal++ user to connect to {es}: +uses the native ++{beat_default_index_prefix}_internal++ user to connect to {es}: + ["source","js",subs="attributes,callouts"] -------------------------------------------------- output.elasticsearch: hosts: ["localhost:9200"] - index: "{beatname_lc}" - username: "{beatname_lc}_internal" + index: "{beat_default_index_prefix}" + username: "{beat_default_index_prefix}_internal" password: "x-pack-test-password" -------------------------------------------------- @@ -86,7 +86,7 @@ output.elasticsearch: -------------------------------------------------- output.elasticsearch: hosts: ["localhost:9200"] - index: "{beatname_lc}" + index: "{beat_default_index_prefix}" ssl.certificate: "/etc/pki/client/cert.pem" <1> ssl.key: "/etc/pki/client/cert.key" -------------------------------------------------- diff --git a/libbeat/docs/security/user-access.asciidoc b/libbeat/docs/security/user-access.asciidoc index 28626afc677..e2a6d53bf5e 100644 --- a/libbeat/docs/security/user-access.asciidoc +++ b/libbeat/docs/security/user-access.asciidoc @@ -8,39 +8,39 @@ and `view_index_metadata` privileges on the {beatname_uc} indices: . Create a role that has the `read` and `view_index_metadata` privileges for the {beatname_uc} indices. You can create roles from the **Management > Roles** UI in {kib} or through the `role` API. -For example, the following request creates a ++{beatname_lc}_reader++ +For example, the following request creates a ++{beat_default_index_prefix}_reader++ role: + -- ["source","sh",subs="attributes,callouts"] --------------------------------------------------------------- -POST _xpack/security/role/{beatname_lc}_reader +POST _xpack/security/role/{beat_default_index_prefix}_reader { "indices": [ { - "names": [ "{beatname_lc}-*" ], <1> + "names": [ "{beat_default_index_prefix}-*" ], <1> "privileges": ["read","view_index_metadata"] } ] } --------------------------------------------------------------- <1> If you use a custom {beatname_uc} index pattern, specify that pattern -instead of the default ++{beatname_lc}-*++ pattern. +instead of the default ++{beat_default_index_prefix}-*++ pattern. -- . Assign your users the reader role so they can access the {beatname_uc} indices: .. If you're using the `native` realm, you can assign roles with the **Management > Users** UI in {kib} or through the `user` API. For -example, the following request grants ++{beatname_lc}_user++ the -++{beatname_lc}_reader++ role: +example, the following request grants ++{beat_default_index_prefix}_user++ the +++{beat_default_index_prefix}_reader++ role: + -- ["source", "sh", subs="attributes,callouts"] --------------------------------------------------------------- -POST /_xpack/security/user/{beatname_lc}_user +POST /_xpack/security/user/{beat_default_index_prefix}_user { "password" : "x-pack-test-password", - "roles" : [ "{beatname_lc}_reader"], + "roles" : [ "{beat_default_index_prefix}_reader"], "full_name" : "{beatname_uc} User" } --------------------------------------------------------------- @@ -48,12 +48,12 @@ POST /_xpack/security/user/{beatname_lc}_user .. If you're using the LDAP, Active Directory, or PKI realms, you assign the roles in the `role_mapping.yml` configuration file. For example, the following snippet grants ++{beatname_uc} User++ -the ++{beatname_lc}_reader++ role: +the ++{beat_default_index_prefix}_reader++ role: + -- ["source", "yaml", subs="attributes,callouts"] --------------------------------------------------------------- -{beatname_lc}_reader: +{beat_default_index_prefix}_reader: - "cn={beatname_uc} User,dc=example,dc=com" --------------------------------------------------------------- For more information, see diff --git a/libbeat/docs/shared-template-load.asciidoc b/libbeat/docs/shared-template-load.asciidoc index 3677a1091a1..7312da1f06c 100644 --- a/libbeat/docs/shared-template-load.asciidoc +++ b/libbeat/docs/shared-template-load.asciidoc @@ -211,7 +211,7 @@ PS > Invoke-RestMethod -Method Delete "http://localhost:9200/{beatname_lc}-*" ---------------------------------------------------------------------- -This command deletes all indices that match the pattern +{beatname_lc}-*+. +This command deletes all indices that match the pattern +{beat_default_index_prefix}-*+. Before running this command, make sure you want to delete all indices that match the pattern. From 5344b7bdb3a175c021f9cd060aee37284ae24fd6 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Mon, 5 Feb 2018 15:05:43 +0100 Subject: [PATCH 21/24] Revert 'warn' -> 'warning' change. --- libbeat/docs/loggingconfig.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libbeat/docs/loggingconfig.asciidoc b/libbeat/docs/loggingconfig.asciidoc index 76c578de8d6..94536f34c38 100644 --- a/libbeat/docs/loggingconfig.asciidoc +++ b/libbeat/docs/loggingconfig.asciidoc @@ -63,7 +63,7 @@ errors, there will be no log file in the directory specified for logs. [[level]] ==== `logging.level` -Minimum log level. One of `debug`, `info`, `warn`, or `error`. The default +Minimum log level. One of `debug`, `info`, `warning`, or `error`. The default log level is `info`. `debug`:: Logs debug messages, including a detailed printout of all events @@ -76,7 +76,7 @@ for all components. `info`:: Logs informational messages, including the number of events that are published. Also logs any warnings, errors, or critical errors. -`warn`:: Logs warnings, errors, and critical errors. +`warning`:: Logs warnings, errors, and critical errors. `error`:: Logs errors and critical errors. From 6ab153b30f351ad53b02b52904b3126841817c48 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Fri, 9 Feb 2018 13:35:58 +0100 Subject: [PATCH 22/24] Add missing quotes. --- libbeat/docs/shared-kibana-config.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libbeat/docs/shared-kibana-config.asciidoc b/libbeat/docs/shared-kibana-config.asciidoc index 8b1f79721bb..d373a5bc0b3 100644 --- a/libbeat/docs/shared-kibana-config.asciidoc +++ b/libbeat/docs/shared-kibana-config.asciidoc @@ -12,12 +12,12 @@ [[setup-kibana-endpoint]] == Set up the Kibana endpoint -ifeval::["{beatname_lc} == "apm-server"] +ifeval::["{beatname_lc}" == "apm-server"] The Kibana dashboards are loaded into Kibana via the Kibana API. This requires a Kibana endpoint configuration. endif::[] -ifeval::["{beatname_lc} != "apm-server"] +ifeval::["{beatname_lc}" != "apm-server"] Starting with {beatname_uc} 6.0.0, the Kibana dashboards are loaded into Kibana via the Kibana API. This requires a Kibana endpoint configuration. endif::[] From 9232834390584a61eb7addd90f90d126e7ea1eb7 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Sun, 11 Feb 2018 09:33:14 +0000 Subject: [PATCH 23/24] Fix conditionals --- libbeat/docs/outputconfig.asciidoc | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index be2fdb1a558..ff0d183ed1e 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -285,6 +285,7 @@ After the specified number of retries, the events are typically dropped. ifeval::["{beatname_lc}" == "filebeat"] Filebeat will ignore the `max_retries` setting and retry until all events are published. +endif::[] ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. endif::[] @@ -532,6 +533,7 @@ After the specified number of retries, the events are typically dropped. ifeval::["{beatname_lc}" == "filebeat"] Filebeat will ignore the `max_retries` setting and retry until all events are published. +endif::[] ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. endif::[] @@ -726,6 +728,8 @@ After the specified number of retries, the events are typically dropped. ifeval::["{beatname_lc}" == "filebeat"] Filebeat will ignore the `max_retries` setting and retry until all events are published. +endif::[] + ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. endif::[] @@ -951,6 +955,7 @@ After the specified number of retries, the events are typically dropped. ifeval::["{beatname_lc}" == "filebeat"] Filebeat will ignore the `max_retries` setting and retry until all events are published. +endif::[] ifeval::["{beatname_lc}" != "filebeat"] Set `max_retries` to a value less than 0 to retry until all events are published. endif::[] From 5a4de2bb0311da9b9a6f45a31e86205c5bcca020 Mon Sep 17 00:00:00 2001 From: Ron Cohen Date: Mon, 19 Feb 2018 11:15:05 +0000 Subject: [PATCH 24/24] Fixed according to @dedemorton's comments. --- libbeat/docs/command-reference.asciidoc | 4 +++- libbeat/docs/outputconfig.asciidoc | 4 ++-- libbeat/docs/shared-ssl-config.asciidoc | 2 +- libbeat/docs/template-config.asciidoc | 2 +- 4 files changed, 7 insertions(+), 5 deletions(-) diff --git a/libbeat/docs/command-reference.asciidoc b/libbeat/docs/command-reference.asciidoc index ac5c96df767..9a21516aced 100644 --- a/libbeat/docs/command-reference.asciidoc +++ b/libbeat/docs/command-reference.asciidoc @@ -22,7 +22,9 @@ ifeval::["{has_ml_jobs}"=="yes"] :setup-command-short-desc: Sets up the initial environment, including the index template, Kibana dashboards (when available), and machine learning jobs (when available) -else::[] +endif::[] + +ifeval::["{has_ml_jobs}"!="yes"] :setup-command-short-desc: Sets up the initial environment, including the index template, Kibana dashboards (when available) endif::[] diff --git a/libbeat/docs/outputconfig.asciidoc b/libbeat/docs/outputconfig.asciidoc index ff0d183ed1e..3453207b090 100644 --- a/libbeat/docs/outputconfig.asciidoc +++ b/libbeat/docs/outputconfig.asciidoc @@ -14,8 +14,8 @@ == Configure the output ifdef::only-elasticsearch[] -You configure {beatname_uc} to write to Elasticsearch by setting options in -the `output.elasticsearch` of the +{beatname_lc}.yml+ config file. +You configure {beatname_uc} to write to Elasticsearch by setting options +in the `output.elasticsearch` section of the +{beatname_lc}.yml+ config file endif::[] ifndef::only-elasticsearch[] diff --git a/libbeat/docs/shared-ssl-config.asciidoc b/libbeat/docs/shared-ssl-config.asciidoc index c1111d83d6c..e813bedf909 100644 --- a/libbeat/docs/shared-ssl-config.asciidoc +++ b/libbeat/docs/shared-ssl-config.asciidoc @@ -1,5 +1,5 @@ [[configuration-ssl]] -== SSL settings for outputs +== Specify SSL settings You can specify SSL options for any <> that supports SSL. You can also specify SSL options when you diff --git a/libbeat/docs/template-config.asciidoc b/libbeat/docs/template-config.asciidoc index 6b82e1e0856..68e5f5612af 100644 --- a/libbeat/docs/template-config.asciidoc +++ b/libbeat/docs/template-config.asciidoc @@ -23,7 +23,7 @@ existing one. you must <>. *`setup.template.name`*:: The name of the template. The default is -+{beatname_lc}-*+. The {beatname_uc} version is always appended to the given ++{beatname_lc}+. The {beatname_uc} version is always appended to the given name, so the final name is +{beatname_lc}-%\{[beat.version]\}+. // Maintainers: a backslash character is required to escape curly braces and