[Upgrade Assistant] logsdb callout

[jloleysens]

Summary

Close https://github.com/elastic/kibana-team/issues/1358

Last step in the Upgrade Assistant has additional calllout:

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[florent-leborgne]

Left copy suggestions. It's still wordy, but still much shorter

[marshallmain]

This looks good to me overall. One question about the version of the docs that we link to. Also I think the decision to enable logsdb by default is being revisited again by the ES team, if it's not enabled by default then we won't need this warning - so we can delay merging or potentially revert later.

[jloleysens]

@marshallmain , I don't mind waiting before merging this. Is there an issue/place where the discussion is happening so I can follow along?

[bitzandeb]

Changes LGTM - we can iterate for 8.19 to find a better place for this.

[tylerperk]

We're still working on the final decision for whether logsdb will be pushed as default for all clusters upgrading to 9.0 or only for new clusters. If the latter, you may not need this at all. If you keep this notice, I made some suggestions to the proposed text. If this will be shown to all upgrading users I'd like to make it clear we're only suggesting they "disable" (aka "don't enable") if they're not already using it.

[tylerperk]

Suggested change

defaultMessage="We recommend that Elastic Security users disable logsdb index mode before upgrading ({logsDataStreamDocs}). The logsdb index mode requires extra CPU capacity during the data indexing process. To avoid timeouts and errors for your data ingestion backups and security detection rules, you should disable the logsdb index mode unless you are sure enough additional hot data tier CPU capacity has been provisioned. You can enable it again after the upgrade ({indexModeWithSecuritySoln})."

defaultMessage="We recommend that Elastic Security users disable logsdb index mode before upgrading ({logsDataStreamDocs}), if not already using logsdb. The logsdb index mode requires extra CPU capacity during the data indexing process. To avoid timeouts and errors for your security detection rules and/or data ingestion backups, you should disable the logsdb index mode unless you are sure enough additional hot data tier CPU capacity has been provisioned. You can enable it again later after the upgrade ({indexModeWithSecuritySoln})."

[jloleysens]

++ that sounds good to me @tylerperk. I'm a little concerned that showing this to all users is suboptimal.

I'm hoping we can conditionally show this to users who have this setting enabled. I'm not super familiar with the functionality, but if we do want to go ahead is there a simple way for us to check for that setting?

[bitzandeb]

@jloleysens - updated copy below for the callout however we will need to update the percent metrics prior to release as these numbers aren't solidified (aligned with @uric, cc: @MikePaquette, @florent-leborgne for review)

@martijnvg @florent-leborgne - is there existing documentation/instructions for how to disable cluster.index.mode? trying to understand where that lives/what we would link to.

Action needed: review new default setting - logsdb index mode

logsdb is a new index mode (learn more) that is applied to logs-*-* data streams by default in Elasticsearch 9.0. logsdb index mode results in significant storage savings (up to 70%) however adds up to X% CPU overhead during data ingestion. In 9.0, logsdb will be applied to newly ingested logs data . If your cluster's hot tier CPU utilization is high (more than Y% on average), it is recommended that you disable this default behavior prior to upgrading to avoid maxing out your hot tier CPU and the downstream issues it may cause. To disable, follow these steps (link). This setting can be changed after upgrade if desired.

stale

[florent-leborgne]

is there existing documentation/instructions for how to disable cluster.index.mode? trying to understand where that lives/what we would link to.

@shainaraskas would you know?

[marshallmain]

We'll have instructions for disabling in the security solution docs (elastic/security-docs#6409), but having it documented elsewhere would be beneficial too imo. I described one way to disable the setting in the linked issue.

[shainaraskas]

We don't have this documented, as far as I can tell. I think the most appropriate home would be the logs data stream docs.

As an aside, it looks like we don't have cluster.logsdb.enabled in the settings reference and might want to do that as well.

[marciw]

We also have this security-specific doc https://www.elastic.co/guide/en/security/current/detections-logsdb-index-mode-impact.html

[jloleysens]

Addressed PR feedback so far. Holding off on merging until we hear from relevant stakeholders and have values for our placeholders.

[jloleysens]

Updated copy based on latest feedback. Still need values for X and Y.

[martijnvg]

is there existing documentation/instructions for how to disable cluster.index.mode? trying to understand where that lives/what we would link to.

@bitzandeb The setting enable or disable logsdb is called cluster.logsdb.enabled. This is a boolean setting. If this is enabled then all data streams matching with the logs-*-* pattern will use logsdb by default (if no index.mode index setting has been defined). We don't have docs about this yet. We're going to add docs about upgrading to logsdb soon: https://www.elastic.co/guide/en/elasticsearch/reference/master/logs-data-stream.html#upgrade-to-logsdb-notes

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 1afdd66

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
aiAssistantManagementSelection	97.5KB	97.7KB	+186.0B
lists	146.9KB	147.1KB	+186.0B
upgradeAssistant	196.5KB	198.1KB	+1.6KB
total			+2.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
core	449.4KB	449.5KB	+186.0B

History

• 💚 Build #274571 succeeded 8956131
• 💚 Build #274341 succeeded ae9f357
• 💚 Build #273094 succeeded 1d4f0ca
• 💛 Build #272590 was flaky 6bcd2c1
• 💚 Build #272051 succeeded a67a187
• 💚 Build #269161 succeeded 6f9302b

cc @jloleysens

[tylerperk]

LGTM thanks