Update output config docs

libbeat/docs/outputconfig.asciidoc

@@ -106,10 +106,6 @@ The number of workers per configured host publishing events to Elasticsearch. Th
 is best used with load balancing mode enabled. Example: If you have 2 hosts and
 3 workers, in total 6 workers are started (3 for each host).
 
-===== port
-
-The default port of the Elasticsearch server if the port number is missing in <<hosts-option>> URL. The default port number is 9200.
-
 ===== username
 
 The basic authentication username for connecting to Elasticsearch.
@@ -148,9 +144,51 @@ for more information about the environment variables.
 
 ===== index
 
-The index root name to write events to. The default is the Beat name.
-For example "{beatname_lc}" generates "[{beatname_lc}-]YYYY.MM.DD" indexes (for example,
-"{beatname_lc}-2015.04.26").
+The index name to write events to. The default is "{beatname_lc}-%{+yyyy.MM.dd}" (for example, "{beatname_lc}-2015.04.26").
+
+===== indices
+
+Array of index selector rules supporting conditionals, format string
+based field access and name mappings. The first rule matching will be used to
+set the `index` for the event to be published. If `indices` is missing or no
+rule matches, the `index` field will be used.
+
+Rule settings:
+
+*`index`*: The index format string to use. If the fields used are missing, the rule fails.
+
+*`mapping`*: Dictionary mapping index names to new names
+
+*`default`*: Default string value if `mapping` does not find a match.
+
+*`when`*: Condition which must succeed in order to execute the current rule.
+
+Examples elasticsearch output with `indices`:
+
+["source","yaml"]
+------------------------------------------------------------------------------
+output.elasticsearch:
+  hosts: ["http://localhost:9200"]
+  index: "logs-%{+yyyy.MM.dd}"
+  indices:
+    - index: "critical-%{+yyyy.MM.dd}"
+      when.contains:
+        message: "CRITICAL"
+    - index: "error-%{+yyyy.MM.dd}"
+      when.contains:
+        message: "ERR"
+------------------------------------------------------------------------------
+
+===== pipeline
+
+format string value configuring the ingest node pipeline to write events to.
+
+===== pipelines
+
+Array of pipeline selector configurations supporting conditionals, format string
+based field access and name mappings. The first rule matching will be used to
+set the `pipeline` for the event to be published. If `pipelines` is missing or no
+rule matches, the `pipeline` field will be used.
 
 ===== template
 
@@ -530,6 +568,54 @@ The password for connecting to Kafka.
 The Kafka topic used for produced events. The setting can be a format string
 using any event field. To set the topic from document type use `%{[type]}`.
 
+===== topics
+
+Array of topic selector rules supporting conditionals, format string
+based field access and name mappings. The first rule matching will be used to
+set the `topic` for the event to be published. If `topics` is missing or no
+rule matches, the `topic` field will be used.
+
+Rule settings:
+
+*`topic`*: The topic format string to use. If the fields used are missing, the
+ rule fails.
+
+*`mapping`*: Dictionary mapping index names to new names
+
+*`default`*: Default string value if `mapping` does not find a match.
+
+*`when`*: Condition which must succeed in order to execute the current rule.
+
+===== key
+
+Optional Kafka event key. If configured, the event key must be unique and can be extracted from the event using a format string.
+
+===== partition
+
+Kafka output broker event partitioning strategy. Must be one of `random`,
+`round_robin`, or `hash`. By default the `hash` partitioner is used.
+
+*`random.group_events`*: Sets the number of events to be published to the same
+ partition, before the partitioner selects a new partition by random. The
+ default value is 1 meaning after each event a new parition is picked randomly.
+
+*`round_robin.group_events`*: Sets the number of events to be published to the
+ same partition, before the partitioner selects the next partition. The default
+ value is 1 meaning after each event the next partition will be selected.
+
+*`hash.hash`*: List of fields used to compute the partitioning hash value from.
+ If no field is configured, the events `key` value will be used.
+
+*`hash.random`*: Randomly distribute events if no hash or key value can be computed.
+
+All partitioners will try to publish events to all partitions by default. If a
+partition's leader becomes unreachable for the beat, the output might block. All
+partitioners support setting `reachable_only` to overwrite this
+behavior. If `reachable_only` is set to `true`, events will be published to
+available partitions only.
+
+NOTE: Publishing to a subset of available partitions potentially increases resource usage because events may become unevenly distributed.
+
 ===== client_id
 
 The configurable ClientID used for logging, debugging, and auditing purposes. The default is "beats".
@@ -662,6 +748,57 @@ The name of the Redis list or channel the events are published to. The default i
 The name of the Redis list or channel the events are published to. The default is
 "{beatname_lc}".
 
+The redis key can be set dynamically using a format string accessing any
+fields in the event to be published.
+
+This configuration will use the `fields.list` field to set the redis list key. If
+`fields.list` is missing, `fallback` will be used.
+
+["source","yaml"]
+------------------------------------------------------------------------------
+output.redis:
+  hosts: ["localhost"]
+  key: "%{[fields.list]:fallback}"
+------------------------------------------------------------------------------
+
+===== keys
+
+Array of key selector configurations supporting conditionals, format string
+based field access and name mappings. The first rule matching will be used to
+set the `key` for the event to be published. If `keys` is missing or no
+rule matches, the `key` field will be used.
+
+Rule settings:
+
+*`key`*: The key format string. If the fields used in the format string are missing, the rule fails.
+
+*`mapping`*: Dictionary mapping key values to new names
+
+*`default`*: Default string value if `mapping` does not find a match.
+
+*`when`*: Condition which must succeed in order to execute the current rule.
+
+Example `keys` settings:
+
+["source","yaml"]
+------------------------------------------------------------------------------
+output.redis:
+  hosts: ["localhost"]
+  key: "default_list"
+  keys:
+    - key: "info_list"   # send to info_list if `message` field contains INFO
+      when.contains:
+        message: "INFO"
+    - key: "debug_list"  # send to debug_list if `message` field contains DEBUG
+      when.contains:
+        message: "DEBUG"
+    - key: "%{[type]}"
+      mapping:
+        "http": "frontend_list"
+        "nginx": "frontend_list"
+        "mysql": "backend_list"
+------------------------------------------------------------------------------
+
 ===== password
 
 The password to authenticate with. The default is no authentication.