diff --git a/docs/configuring.asciidoc b/docs/configuring.asciidoc index 0def3f96690..390de5b4c97 100644 --- a/docs/configuring.asciidoc +++ b/docs/configuring.asciidoc @@ -27,7 +27,7 @@ include::./copied-from-beats/shared-ssl-config.asciidoc[] include::./template-config.asciidoc[] -include::./ilm-setup.asciidoc[] +include::./ilm.asciidoc[] include::./copied-from-beats/loggingconfig.asciidoc[] diff --git a/docs/copied-from-beats/command-reference.asciidoc b/docs/copied-from-beats/command-reference.asciidoc index e1ed7c3146f..d456c595ce8 100644 --- a/docs/copied-from-beats/command-reference.asciidoc +++ b/docs/copied-from-beats/command-reference.asciidoc @@ -636,8 +636,15 @@ enabled modules in the +{beatname_lc}.yml+ file. If you used the directory, also specify the `--modules` flag. endif::[] +ifndef::apm-server[] *`--template`*:: Sets up the index template only. +endif::apm-server[] + +ifdef::apm-server[] +*` --index-management`*:: +Sets up <>. +endif::[] {global-flags} diff --git a/docs/guide/apm-release-notes.asciidoc b/docs/guide/apm-release-notes.asciidoc index 3c9bb13eb7a..73be25c1302 100644 --- a/docs/guide/apm-release-notes.asciidoc +++ b/docs/guide/apm-release-notes.asciidoc @@ -7,6 +7,7 @@ For a full list of changes, see the {apm-server-ref-v}/release-notes.html[APM Server Release Notes] or the {kibana-ref}/release-notes.html[Kibana Release Notes]. +* <> * <> * <> * <> @@ -16,6 +17,16 @@ For a full list of changes, see the * <> * <> +[[release-highlights-7.2.0]] +=== APM version 7.2.0 + +[float] +==== New features + +*APM Server* + +* Support for {apm-server-ref-v}/ilm.html[index lifecycle management (ILM)] + [[release-highlights-7.1.0]] === APM version 7.1.0 diff --git a/docs/ilm-setup.asciidoc b/docs/ilm-setup.asciidoc index 231dc3ab7ee..6c80e267f4b 100644 --- a/docs/ilm-setup.asciidoc +++ b/docs/ilm-setup.asciidoc @@ -1,13 +1,9 @@ [role="xpack"] [[manual-ilm-setup]] -== Manual index lifecycle management +=== Manual index lifecycle management -APM Server can take advantage of the index lifecycle management (ILM) capabilities of Elasticsearch. -ILM enables you to automate how you want to manage your indices over time. -You can base actions on factors such as shard size and performance requirements. - -The guide below will help you set up a custom ILM policy for span indices. -You can repeat the actions for any other indices. +The guide below will help you set up a custom index lifecycle management policy for span indices. +You can repeat these actions for any other indices (transactions, errors, etc.). NOTE: If you're migrating from an existing setup, any indices present before ILM was configured will need to be managed manually. diff --git a/docs/ilm.asciidoc b/docs/ilm.asciidoc new file mode 100644 index 00000000000..bd51490aaf6 --- /dev/null +++ b/docs/ilm.asciidoc @@ -0,0 +1,131 @@ +[[ilm]] +[role="xpack"] +== Index lifecycle management (ILM) + +Use the {ref}/getting-started-index-lifecycle-management.html[index lifecycle management (ILM)] +feature in {es} to manage your APM Server indices as they age. +ILM enables you to automate how you want to manage your indices over time, +by automating rollovers to a new index when the existing index reaches a specified size or age. + +You can view and edit the index lifecycle policies in the *Index lifecycle policies* UI in {kib}. +For more information about working with the UI, +see {kibana-ref}/index-lifecycle-policies.html[Index lifecycle policies]. + +APM Server currently offers two ways to get started with ILM: + +1. <> - Get up and running with a default index lifecycle management policy as quickly as possible. +2. <> - Customize and manage your own ILM policies. + +NOTE: If you're migrating from an existing setup, +any indices present before ILM was configured will need to be managed manually. + +[[ilm-default]] +=== ILM default policy + +Index lifecycle management will manage an index based on its defined policy. +The default ILM configuration applies hot and warm phase policies. +Cold and delete phases are not defined. +Because errors and spans lose information value faster than metrics and transactions do, +there are two different policies defined: +one for `errors` and `spans`, and one for `metrics` and `transactions`. + +[options="header"] +|======================================================================= +|Event type |Hot |Warm +|Errors & Spans +|Rollover: `max_size: 50gb`, `max_age: 1 day`, +`priority: 100` +|`min_age: 7 days`, +readonly, +`priority: 50` +|Transactions & Metrics +|Rollover: `max_size: 50gb`, `max_age: 7 days`, +`priority: 100` +|`min_age: 31 days`, +readonly, +`priority: 50` +|======================================================================= + +IMPORTANT: Changes to the default index lifecycle policy do not take effect until the current index has rolled over. +If you'd like to manage a custom policy, see <>. + +[float] +[[ilm-setup]] +==== ILM default policy setup + +To set up index lifecycle management, set `ilm.enabled` to `true` in apm-server.yml. +ILM can only be enabled for `output.elasticsearch`. +When enabled, configurations defined for `output.elasticsearch.index` and `output.elasticsearch.indices` will be ignored. + +It is recommended to set up index lifecycle management (ILM) before starting APM Server. +This excludes setup from the ingestion process, which allows you to ensure ILM is set up correctly before using APM. + +Run the <> with the ` --index-management` flag to set up the default ILM policy: + +[source,js] +----------------------- +./apm-server setup --index-management +----------------------- +// CONSOLE + +You can confirm the policy was created with the GET lifecycle policy API. +Here's what the transaction response looks like: + +[source,js] +----------------------- +GET _ilm/policy +{ + "apm-7.2.0-transaction": { + "version": 1, + "modified_date": "2019-05-28T15:55:26.791Z", + "policy": { + "phases": { + "warm": { + "min_age": "31d",<1> + "actions": { + "readonly": {}, + "set_priority": { + "priority": 50 + } + } + }, + "hot": { + "min_age": "0ms", + "actions": { + "rollover": { + "max_size": "50gb",<2> + "max_age": "7d"<3> + }, + "set_priority": { + "priority": 100<4> + } + } + } + } + } + } +} +----------------------- +// CONSOLE +<1> Move to warm phase after _31 days_ +<2> Rollover after _50gb_ +<3> Rollover after _7 days_ +<4> Priority for recovering your indices after a node restart. Higher priorities are recovered first. + +Your indices are now configured to use index lifecycle management. Go ahead and <>. + +NOTE: If you switch between ILM enabled/disabled multiple times, +you should set `setup.template.overwrite` to `true` to ensure a complete setup. + +[float] +==== ILM default policy upgrades + +If you decide to customize the default ILM policy, any customizations will be overwritten when you upgrade APM Server. +Default policies are also subject to change in future releases + +// Policies are versioned so they can change. +// Indices are versioned so they can change. +// An upgrade creates new templates, policies, and indices. +// If you customize anything, it will revert back to the default during an upgrade + +include::./ilm-setup.asciidoc[] \ No newline at end of file