Merge remote-tracking branch 'upstream/master' into kertal-pr-2020-04…

…-02-discover-unskip-field-visualize-functional

.github/CODEOWNERS

@@ -13,6 +13,7 @@
 /src/legacy/core_plugins/kibana/public/dev_tools/ @elastic/kibana-app
 /src/legacy/core_plugins/vis_type_vislib/ @elastic/kibana-app
 /src/plugins/vis_type_xy/ @elastic/kibana-app
+/src/plugins/vis_type_table/ @elastic/kibana-app
 /src/plugins/kibana_legacy/ @elastic/kibana-app
 /src/plugins/vis_type_timelion/ @elastic/kibana-app
 /src/plugins/dashboard/ @elastic/kibana-app

.i18nrc.json

@@ -48,7 +48,7 @@
     "visDefaultEditor": "src/plugins/vis_default_editor",
     "visTypeMarkdown": "src/plugins/vis_type_markdown",
     "visTypeMetric": "src/plugins/vis_type_metric",
-    "visTypeTable": "src/legacy/core_plugins/vis_type_table",
+    "visTypeTable": "src/plugins/vis_type_table",
     "visTypeTagCloud": "src/legacy/core_plugins/vis_type_tagcloud",
     "visTypeTimeseries": ["src/legacy/core_plugins/vis_type_timeseries", "src/plugins/vis_type_timeseries"],
     "visTypeVega": "src/legacy/core_plugins/vis_type_vega",

CONTRIBUTING.md

@@ -22,6 +22,7 @@ A high level overview of our contributing guidelines.
     - [Setting Up SSL](#setting-up-ssl)
   - [Linting](#linting)
   - [Internationalization](#internationalization)
+  - [Localization](#localization)
   - [Testing and Building](#testing-and-building)
     - [Debugging server code](#debugging-server-code)
     - [Instrumenting with Elastic APM](#instrumenting-with-elastic-apm)
@@ -408,6 +409,11 @@ ReactDOM.render(
 There are a number of tools created to support internationalization in Kibana that would allow one to validate internationalized labels,
 extract them to a `JSON` file or integrate translations back to Kibana. To know more, please read corresponding [readme](src/dev/i18n/README.md) file.
 
+### Localization
+
+We cannot support accepting contributions to the translations from any source other than the translators we have engaged to do the work.
+We are still to develop a proper process to accept any contributed translations. We certainly appreciate that people care enough about the localization effort to want to help improve the quality. We aim to build out a more comprehensive localization process for the future and will notify you once contributions can be supported, but for the time being, we are not able to incorporate suggestions.
+
 ### Testing and Building
 
 To ensure that your changes will not break other functionality, please run the test suite and build process before submitting your Pull Request.

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.md

@@ -88,7 +88,6 @@
 |  [SavedQuery](./kibana-plugin-plugins-data-public.savedquery.md) |  |
 |  [SavedQueryService](./kibana-plugin-plugins-data-public.savedqueryservice.md) |  |
 |  [SearchSourceFields](./kibana-plugin-plugins-data-public.searchsourcefields.md) |  |
-|  [SearchStrategyProvider](./kibana-plugin-plugins-data-public.searchstrategyprovider.md) |  |
 |  [TabbedAggColumn](./kibana-plugin-plugins-data-public.tabbedaggcolumn.md) | \* |
 |  [TabbedTable](./kibana-plugin-plugins-data-public.tabbedtable.md) | \* |
 |  [TimeRange](./kibana-plugin-plugins-data-public.timerange.md) |  |

docs/management/advanced-options.asciidoc

@@ -217,6 +217,8 @@ might increase the search time. This setting is off by default. Users must opt-i
 [horizontal]
 `siem:defaultAnomalyScore`:: The threshold above which Machine Learning job anomalies are displayed in the SIEM app.
 `siem:defaultIndex`:: A comma-delimited list of Elasticsearch indices from which the SIEM app collects events.
+`siem:ipReputationLinks`:: A JSON array containing links for verifying the reputation of an IP address. The links are displayed on
+{siem-guide}/siem-ui-overview.html#network-ui[IP detail] pages.
 `siem:enableNewsFeed`:: Enables the security news feed on the SIEM *Overview*
 page.
 `siem:newsFeedUrl`:: The URL from which the security news feed content is

docs/settings/ml-settings.asciidoc

@@ -8,7 +8,6 @@
 You do not need to configure any settings to use {kib} {ml-features}. They are
 enabled by default.
 
-[float]
 [[general-ml-settings-kb]]
 ==== General {ml} settings
 
@@ -19,3 +18,11 @@ If set to `false` in `kibana.yml`, the {ml} icon is hidden in this {kib}
 instance. If `xpack.ml.enabled` is set to `true` in `elasticsearch.yml`, however,
 you can still use the {ml} APIs. To disable {ml} entirely, see the
 {ref}/ml-settings.html[{es} {ml} settings].
+
+[[data-visualizer-settings]]
+==== {data-viz} settings
+
+`xpack.ml.file_data_visualizer.max_file_size`::
+Sets the file size limit when importing data in the {data-viz}. The default
+value is `100MB`. The highest supported value for this setting is `1GB`.
+

docs/setup/production.asciidoc

@@ -133,7 +133,8 @@ server.port
 Settings that must be the same:
 --------
 xpack.security.encryptionKey //decrypting session cookies
-xpack.reporting.encryptionKey //decrypting reports stored in Elasticsearch
+xpack.reporting.encryptionKey //decrypting reports
+xpack.encryptedSavedObjects.encryptionKey // decrypting saved objects
 --------
 
 Separate configuration files can be used from the command line by using the `-c` flag:

docs/siem/siem-ui.asciidoc

@@ -35,7 +35,7 @@ image::siem/images/network-ui.png[]
 
 [float]
 [[detections-ui]]
-=== Detections (Beta)
+=== Detections (beta)
 
 The Detections feature automatically searches for threats and creates 
 signals when they are detected. Signal detection rules define the conditions 
@@ -50,6 +50,22 @@ or the Detections API.
 [role="screenshot"]
 image::siem/images/detections-ui.png[]
 
+[float]
+[[cases-ui]]
+=== Cases (beta)
+
+Cases are used to open and track security issues directly in SIEM. 
+Cases list the original reporter and all users who contribute to a case
+(`participants`). Case comments support Markdown syntax, and allow linking to
+saved Timelines. Additionally, you can send cases to external systems from
+within SIEM (currently ServiceNow).
+
+For information about opening, updating, and closing cases, see
+{siem-guide}/cases-overview.html[Cases] in the SIEM Guide.
+
+[role="screenshot"]
+image::siem/images/cases-ui.png[]
+
 [float]
 [[timelines-ui]]
 === Timeline

docs/user/alerting/action-types/pagerduty.asciidoc

@@ -4,19 +4,142 @@
 
 The PagerDuty action type uses the https://v2.developer.pagerduty.com/docs/events-api-v2[v2 Events API] to trigger, acknowledge, and resolve PagerDuty alerts.
 
+* <<pagerduty-benefits, PagerDuty and Elastic integration benefits>>
+* <<pagerduty-connector-configuration, Connector configuration>>
+* <<pagerduty-action-configuration, Action configuration>>
+
+[float]
+[[pagerduty-benefits]]
+=== PagerDuty + Elastic integration benefits
+
+By integrating PagerDuty with alerts, you can:
+
+* Route your alerts to the right PagerDuty responder within your team, based on your structure, escalation policies, and workflows.
+* Automatically generate incidents of different types and severity based on each alert’s context.
+* Tailor the incident data to match your needs by easily passing the alerting context from Kibana to PagerDuty.
+
+[float]
+[[pagerduty-how-it-works]]
+==== How it works
+
+{kib} allows you to create alerts to notify you of a significant move
+in your dataset.
+You can create alerts for all your Observability, Security, and Elastic Stack use cases.
+Alerts will trigger a new incident on the corresponding PagerDuty service.
+
+[float]
+==== Requirements
+
+In the `kibana.yml` configuration file, you must add the <<general-alert-action-settings, saved objects encryption setting>>.
+This is required to encrypt parameters that must be secured, for example PagerDuty’s integration key.
+
+If you have security enabled:
+
+* You must have
+application privileges to access Metrics, APM, Uptime, or SIEM.
+* If you are using a self-managed deployment with security, you must have
+Transport Security Layer (TLS) enabled for communication <<configuring-tls-kib-es, between Elasticsearch and Kibana>>.
+Alerts uses API keys to secure background alert checks and actions,
+and API keys require {ref}/configuring-tls.html#tls-http[TLS on the HTTP interface].
+
+Although not a requirement, to harden the integrations security you might want to
+review the <<action-settings, Actions settings>> that are available to you.
+
+[float]
+[[pagerduty-support]]
+==== Support
+If you need help with this integration, get in touch with the {kib} team by visiting
+https://support.elastic.co[support.elastic.co] or by using the *Ask Elastic* option in the {kib} Help menu.
+You can also select the {kib} category at https://discuss.elastic.co/[discuss.elastic.co].
+
+[float]
+[[pagerduty-integration-walkthrough]]
+==== Integration with PagerDuty walkthrough
+
+[float]
+[[pagerduty-in-pagerduty]]
+===== In PagerDuty
+
+. From the *Configuration* menu, select *Services*.
+. Add an integration to a service:
++
+* If you are adding your integration to an existing service,
+click the name of the service you want to add the integration to.
+Then, select the *Integrations* tab and click the *New Integration* button.
+* If you are creating a new service for your integration,
+go to
+https://support.pagerduty.com/docs/services-and-integrations#section-configuring-services-and-integrations[Configuring Services and Integrations]
+and follow the steps outlined in the *Create a New Service* section, selecting *Elastic* as the *Integration Type* in step 4.
+Continue with the <<pagerduty-in-elastic, In Elastic>> section once you have finished these steps.
+
+. Enter an *Integration Name* in the format Elastic-service-name (for example, Elastic-Alerting or Kibana-APM-Alerting)
+and select Elastic from the *Integration Type* menu.
+. Click *Add Integration* to save your new integration.
++
+You will be redirected to the *Integrations* tab for your service. An Integration Key is generated on this screen.
++
+[role="screenshot"]
+image::user/alerting/images/pagerduty-integration.png[PagerDuty Integrations tab]
+
+. Save this key, as you will use it when you configure the integration with Elastic in the next section.
+
+[float]
+[[pagerduty-in-elastic]]
+===== In Elastic
+
+. Create a PagerDuty Connector in Kibana.  You can:
++
+* Create a connector as part of creating an alert by selecting PagerDuty in the *Actions*
+section of the alert configuration and selecting *Add new*.
+* Alternatively, create a connector by navigating to *Management* from the {kib} navbar and selecting
+*Alerts and Actions*. Then, select the *Connectors* tab, click the *Create connector* button, and select the PagerDuty option.
+
+. Configure the connector by giving it a name and optionally entering the API URL and Routing Key, or using the defaults.
++
+See <<pagerduty-in-pagerduty, In PagerDuty>> for how to obtain the endpoint and key information from PagerDuty and
+<<pagerduty-connector-configuration, Connector configuration>> for more details.
+
+. Save the Connector.
+
+. Create an alert using *Management > Alerts and Actions* or the application of your choice.
+
+. Set up an action using your PagerDuty connector, by determining:
++
+* The action’s type: Trigger, Resolve, or Acknowledge.
+* The event’s severity: Info, warning, error, or critical.
+* An array of different fields, including the timestamp, group, class, component, and your dedup key.
+Depending on your custom needs, assign them variables from the alerting context.
+To see the available context variables, click on the *Add alert variable* icon next
+to each corresponding field. For more details on these parameters, see the
+<<pagerduty-action-configuration, Actions Configuration>> and the PagerDuty
+https://v2.developer.pagerduty.com/v2/docs/send-an-event-events-api-v2[API v2 documentation].
+
+
+[float]
+[[pagerduty-uninstall]]
+==== How to uninstall
+To remove a PagerDuty connector from an alert, simply remove it
+from the *Actions* section of that alert, using the remove (x) icon.
+This will disable the integration for the particular alert.
+
+To delete the connector entirely, go to *Management > Alerts and Actions*.
+Select the *Connectors* tab, and then click on the delete icon.
+This is an irreversible action and impacts all alerts that use this connector.
+
+
 [float]
 [[pagerduty-connector-configuration]]
-==== Connector configuration
+=== Connector configuration
 
 PagerDuty connectors have the following configuration properties:
 
 Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
-API URL::   An optional PagerDuty event URL. Defaults to `https://events.pagerduty.com/v2/enqueue`. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure the hostname is whitelisted. 
+API URL::   An optional PagerDuty event URL. Defaults to `https://events.pagerduty.com/v2/enqueue`. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure the hostname is whitelisted.
 Routing Key::   A 32 character PagerDuty Integration Key for an integration on a service or on a global ruleset.
 
 [float]
 [[pagerduty-action-configuration]]
-==== Action configuration
+=== Action configuration
 
 PagerDuty actions have the following properties:
 
@@ -26,8 +149,8 @@ Dedup Key::     All actions sharing this key will be associated with the same Pa
 Timestamp::     An *optional* https://v2.developer.pagerduty.com/v2/docs/types#datetime[ISO-8601 format date-time], indicating the time the event was detected or generated.
 Component::     An *optional* value indicating the component of the source machine that is responsible for the event, for example `mysql` or `eth0`.
 Group::         An *optional* value indicating the logical grouping of components of a service, for example `app-stack`.
-Source::        An *optional* value indicating the affected system, preferably a hostname or fully qualified domain name. Defaults to the {kib} saved object id of the action. 
+Source::        An *optional* value indicating the affected system, preferably a hostname or fully qualified domain name. Defaults to the {kib} saved object id of the action.
 Summary::       An *optional* text summary of the event, defaults to `No summary provided`. The maximum length is 1024 characters.
 Class::         An *optional* value indicating the class/type of the event, for example `ping failure` or `cpu load`.
 
-For more details on these properties, see https://v2.developer.pagerduty.com/v2/docs/send-an-event-events-api-v2[PagerDuty v2 event parameters].
+For more details on these properties, see https://v2.developer.pagerduty.com/v2/docs/send-an-event-events-api-v2[PagerDuty v2 event parameters].

docs/user/dashboard.asciidoc

@@ -98,6 +98,24 @@ to the new dimensions.
 * To delete a panel, open the panel menu and select *Delete from dashboard.* Deleting a panel from a
 dashboard does *not* delete the saved visualization or search.
 
+[float]
+[[cloning-a-panel]]
+=== Clone dashboard elements
+
+In *Edit* mode, you can clone any panel on a dashboard.
+
+To clone an existing panel, open the panel menu of the element you wish to clone, then select *Clone panel*.
+
+* Cloned panels appear beside the original, and will move other panels down to make room if necessary.
+
+* Clones support all of the original panel's functionality, including renaming, editing, and cloning.
+
+* All cloned visualizations will appear in the visualization list.
+
+[role="screenshot"]
+image:images/clone_panel.gif[clone panel]
+
+
 [float]
 [[viewing-detailed-information]]
 === Inspect and edit elements

docs/user/ml/index.asciidoc

@@ -4,31 +4,31 @@
 
 [partintro]
 --
-As datasets increase in size and complexity, the human effort required to
+As data sets increase in size and complexity, the human effort required to
 inspect dashboards or maintain rules for spotting infrastructure problems,
 cyber attacks, or business issues becomes impractical. Elastic {ml-features}
 such as {anomaly-detect} and {oldetection} make it easier to notice suspicious
 activities with minimal human interference.
 
-If you have a basic license, you can use the *Data Visualizer* to learn more
-about your data. In particular, if your data is stored in {es} and contains a
-time field, you can use the *Data Visualizer* to identify possible fields for
-{anomaly-detect}:
+{kib} includes a free *{data-viz}* to learn more about your data. In particular,
+if your data is stored in {es} and contains a time field, you can use the
+*{data-viz}* to identify possible fields for {anomaly-detect}:
 
 [role="screenshot"]
-image::user/ml/images/ml-data-visualizer-sample.jpg[Data Visualizer for sample flight data]
+image::user/ml/images/ml-data-visualizer-sample.jpg[{data-viz} for sample flight data]
 
-experimental[] You can also upload a CSV, NDJSON, or log file (up to 100 MB in 
-size). The *Data Visualizer* identifies the file format and field mappings. You 
-can then optionally import that data into an {es} index.
+experimental[] You can also upload a CSV, NDJSON, or log file. The *{data-viz}*
+identifies the file format and field mappings. You can then optionally import
+that data into an {es} index. To change the default file size limit, see
+<<data-visualizer-settings>>.
 
-You need the following permissions to use the Data Visualizer with file upload:
+You need the following permissions to use the {data-viz} with file upload:
 
 * cluster privileges: `monitor`, `manage_ingest_pipelines`
 * index privileges: `read`, `manage`, `index`
 
 For more information, see {ref}/security-privileges.html[Security privileges] 
-and {ref}/built-in-roles.html[Built-in roles].
+and {ml-docs}/setup.html[Set up {ml-features}].
 
 --