Skip to content

Latest commit

 

History

History
309 lines (232 loc) · 10.4 KB

troubleshooting.asciidoc

File metadata and controls

309 lines (232 loc) · 10.4 KB

Troubleshooting

beta[]

We have collected the most common known problems and frequently asked questions here. If your question isn’t answered here, please review open issues in the following GitHub repositories:

Contact us in the {im-forum}[discuss forum]. Your feedback is very valuable to us.

Common problems:

Frequently asked questions:

{ingest-manager} is not listed in the {kib} side navigation

The {ingest-manager} app is enabled by default. If you are unable to see the app in {kib}, make sure it’s enabled.

To enable {ingest-manager} on {ecloud}:

  1. Go to your deployment in the user console.

  2. Under the deployment name in the side navigation, click Edit.

  3. In the {kib} section, expand User setting overrides and enter the following setting:

    xpack.ingestManager.enabled: true

  4. Click Save.

{kib} will restart automatically. When {kib} is available, refresh the browser to see the {ingest-manager} app in the navigation menu.

To enable {ingest-manager} on a self-managed cluster:

  1. In the {es} configuration file, config/elasticsearch.yml, set the following security settings to enable security and API keys:

    xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true

  2. In the {kib} configuration file, config/kibana.yml, enable {ingest-manager} and specify user credentials:

    xpack.ingestManager.enabled: true
xpack.ingestManager.fleet.tlsCheckDisabled: true (1)
xpack.security.enabled: true
elasticsearch.username: "elastic" (2)
elasticsearch.password: "abc123iUnbRftkABC123"

    1. This setting is not required if you configure TLS checking.

    2. Specify a user who is authorized to use {ingest-manager}.

Tip

To set up passwords, you can use the documented {es} APIs or the elasticsearch-setup-passwords command. For example:

./bin/elasticsearch-setup-passwords auto

After running the command, copy the Elastic user name to the {kib} config file. Then restart {kib}.

The /api/ingest_management/setup endpoint returns an error because it can’t reach the package registry

In order to install {integrations}, the {ingest-manager} app needs to connect to an external service called the {package-registry}. For this to work, the {kib} server must be able to connect to https://epr.elastic.co on port 443.

{ingest-manager} in {kib} crashes

To find more about the error, open your browser’s development console, navigate to the Network tab, and refresh the page. One of the requests to the {ingest-manager} API will most likely have returned an error. If the error message doesn’t give you enough information to fix the problem, please contact us in the {im-forum}[discuss forum].

{agent} enrollment fails on the host with Client.Timeout exceeded message

{agent} must be able to connect to the {kib} instance to enroll in {fleet}. If the agent is unable to connect, you will see the following failure:

fail to enroll: fail to execute request to {kib}:Post http://kibana:5601/api/ingest_manager/fleet/agents/enroll?: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

This may occur if the host is unable to connect to {kib}. To troubleshoot the problem:

  1. Check for networking problems. Run the ping command from the host to confirm that it can reach the {kib} instance.

  2. Verify that the URL and port you specified during enrollment are correct for your environment.

  3. Check the enrollment key that you specified during enrollment to confirm that the key is valid. To do this:

    1. In {ingest-manager}, go to the {fleet} tab and click Enrollment Tokens.

    2. Click the eyeball icon to see the secret. The secret should match the string that you used to enroll {agent} on your host.

    3. If the secret doesn’t match, create a new enrollment token and use the new token when you run the elastic-agent enroll command.

{es} authentication service fails with Authentication using apikey failed message

{fleet} requires an encryption key in order to save API keys and encrypt them in {kib}. To provide an API key, set the xpack.encryptedSavedObjects.encryptionKey property in the kibana.yml configuration file. For example:

xpack.encryptedSavedObjects.encryptionKey: "something_at_least_32_characters"

{agent} fails with Agent process is not root/admin or validation failed message

Make sure the user running {agent} has root privileges. If you’re running {agent} in the foreground (and not as a service) on Linux or macOS, run the agent under the root user, for example, sudo or su. Some integrations require root privileges to collect sensitive data.

If you’re using the {elastic-endpoint} integration, also make sure you’re running {agent} under the SYSTEM account.

Tip
 If you install {agent} as a service as described in [elastic-agent-installation], {agent} runs under the SYSTEM account by default.

To run {agent} under the SYSTEM account, you can:

  1. Download PsExec and extract the contents to a folder, for example, d:\tools.

  2. Open a command prompt as an Administrator (right-click the Command Prompt icon and select Run As Administrator).

  3. From the command prompt, run {agent} under the SYSTEM account:

    d:\tools\psexec.exe -sid "C:\Program Files\Elastic-Agent\elastic-agent.exe" run

{agent} hangs while unenrolling

When you unenroll an agent, {fleet} waits for acknowledgement from the agent before it completes the unenrollment process. If {fleet} doesn’t receive acknowledgement, the status hangs at unenrolling.

If this happens, select Force unenroll from the Actions menu in {fleet}.

This will invalidate all API keys related to the agent and change the status to inactive so that the agent no longer appears in {fleet}.

Why doesn’t my enrolled Agent show up in the {ingest-manager} app?

If {agent} was successfully enrolled, but doesn’t show up in the {fleet} list, it might not be started. You need to start {agent}.

Where does {agent} store logs after startup?

When started successfully, {metricbeat} logs are stored in data/logs/metricbeat under the folder where {agent} was started. If that log path does not exist, {agent} was unable to start {metricbeat}, which is a higher level problem to triage.

What configuration is the {agent} running?

To find the configuration file, inspect the elastic-agent.yml file in the folder where you ran {agent}. If you’re running the agent in {fleet} mode, this file contains the following citation:

Management: mode: "fleet"

The action_store.yml contains the entire, unencrypted configuration:

  • To see the {es} location, look at outputs:hosts.

  • To see the {agent} version, look at the download folder and zip filenames.

This file also shows the version of all packages used by the current configuration.

Why can’t I see the data {agent} is sending?

If {elastic-agent} is set up and running, but you don’t see data in {kib}:

  1. Go to Management > Dev Tools in {kib}, and in the Console, search your index for data. For example:

    GET metrics-*/_search

    Or if you prefer, go to the Discover app.

  2. Look at the data that {elastic-agent} has sent and see if the name.host field contains your host machine name.

If you don’t see data for your host, it’s possible that the data is blocked in the network, or that a firewall or security problem is preventing the {agent} from sending the data.

Although it’s redundant to install stand-alone {metricbeat}, you might want to try installing it to see if it’s able to send data successfully to {es}. For more information, see {metricbeat-ref}/metricbeat-installation-configuration.html[{metricbeat} quick start].

If {metricbeat} is able to send data to {es}, there is possibly a bug or problem with {agent}, and you should report it.

How do I restore an {agent} that I deleted from {fleet}?

It’s ok, we’ve got your back! The data is still in {es}. To add {agent} to {fleet} again, [stop-elastic-agent], re-enroll it on the host, then run {agent}.

How do I restart {agent} after rebooting my host?

On Windows, if you used Powershell to install {agent} as a service, the agent should still be running after rebooting the host.

On macOS and Linux, you need to restart {agent} from the command line after rebooting the host.

Support for installing {agent} as a service on all supported systems will be available in a future release. To achieve this in the meantime, you can add the start command to a user’s startup profile.

What is the Endpoint integration shown in {ingest-manager}?

In 7.8, the Endpoint integration is non-functional. It cannot be used yet. It exists as an artifact of the current feature development. Please watch for announcements during upcoming release cycles. As a teaser, Endpoint is the integration that will allow the Elastic Security app to have a dedicated executable running like {beats} to protect the host and respond to detected security concerns. Endpoint will be managed by {agent} in the same way that {beats} are managed.