Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade ES client to 9.0.0-alpha.3 #208776

Merged
merged 78 commits into from
Feb 25, 2025
Merged

Upgrade ES client to 9.0.0-alpha.3 #208776

merged 78 commits into from
Feb 25, 2025

Conversation

afharo
Copy link
Member

@afharo afharo commented Jan 29, 2025

Summary

Updating the ES client to 9.0.

Resolves #116102

What changes?

Breaking change: body has been removed.

Most of the changes are about bringing all the content inside the body as a root attribute to the API params:

const response = await client.search({
  index: 'test',
-  body: {
    query: {
      match_all: {}
    }
-  }
})

For this reason, enabling the "Hide whitespace changes" option when reviewing is recommended.

Some exceptions to this rule:

  • Bulk APIs replace the body array with operations array (direct replacement)
  • Index Put Settings API replace body array with settings (direct replacement)
  • Msearch replaces the body array with searches array (direct replacement)
  • Document Index API replaces body with document (direct replacement)
  • Create Repository replaces body with repository (direct replacement)

Because of a known issue in the client (elastic/elasticsearch-js#2584), there's still an escape hatch to send data in the body in case the specific use case requires it via // @ts-expect-error [email protected] https://github.com/elastic/elasticsearch-js/issues/2584, but it shouldn't be abused because we lose types. In this PR we've used it in those scenarios where we reuse the response of a GET as the body of a PUT/POST.

Other changes

  • estypes can be imported from the root of the library as import type { estypes } from '@elastic/elasticsearch';
  • estypesWithBody have been removed
  • requestTimeout's 30s default has been removed in the client. This PR explicitly adds the setting in all client usages.

Identify risks

# Conflicts:
#	x-pack/solutions/security/packages/data-stream-adapter/src/create_or_update_data_stream.ts
#	x-pack/solutions/security/packages/index-adapter/src/create_or_update_index.ts
Copy link
Contributor

github-actions bot commented Feb 3, 2025

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

@afharo afharo changed the title [WIP] Upgrade ES client to 9.0.0-alpha.2 [WIP] Upgrade ES client to 9.0.0-alpha.3 Feb 3, 2025
Copy link
Contributor

github-actions bot commented Feb 3, 2025

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

Copy link
Contributor

@eokoneyo eokoneyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good, tested csv exports with pit and scroll strategy.

Copy link
Contributor

@szaffarano szaffarano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDA team LGTM!

@@ -131,6 +131,7 @@ export function registerSearchApplicationsRoutes({ log, router }: RouteDependenc
name: request.params.engine_name,
search_application: {
indices: request.body.indices,
// @ts-expect-error [email protected] has this parameter been renamed to analytics_collection_name?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually a different one. name is the search application's display name, analytics_collection_name is the name of the related collection
https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/ent-search/src/main/java/org/elasticsearch/xpack/application/search/SearchApplication.java#L67

Copy link
Contributor

@shahzad31 shahzad31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smoke testing synthetics/SLO looks good !!

@afharo afharo enabled auto-merge (squash) February 25, 2025 12:35
@afharo
Copy link
Member Author

afharo commented Feb 25, 2025

MKI tests passed! Running a final CI with the latest main before merging.

@elasticmachine
Copy link
Contributor

elasticmachine commented Feb 25, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
aiops 454.5KB 454.5KB -28.0B
apm 2.3MB 2.3MB -7.0B
dataViewFieldEditor 113.7KB 113.6KB -14.0B
dataVisualizer 616.2KB 616.1KB -99.0B
discover 811.0KB 810.9KB -14.0B
lens 1.6MB 1.6MB -7.0B
ml 5.2MB 5.2MB +110.0B
securitySolution 8.9MB 8.9MB -21.0B
slo 795.6KB 795.6KB -7.0B
stackAlerts 75.3KB 75.3KB -24.0B
synthetics 791.0KB 790.9KB -126.0B
threatIntelligence 754.0KB 753.9KB -63.0B
transform 475.8KB 475.8KB -21.0B
uptime 376.2KB 376.2KB -16.0B
ux 163.8KB 163.7KB -99.0B
visTypeVega 1.9MB 1.9MB -59.0B
total -495.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
data 398.0KB 398.1KB +66.0B
dataViews 61.9KB 61.9KB -7.0B
unifiedDocViewer 11.9KB 11.8KB -48.0B
total +11.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
@kbn/observability-utils-server 1 0 -1
data 56 55 -1
indexManagement 16 15 -1
triggersActionsUi 120 119 -1
total -4

Total ESLint disabled count

id before after diff
@kbn/observability-utils-server 1 0 -1
data 57 56 -1
indexManagement 18 17 -1
triggersActionsUi 125 124 -1
total -4

History

cc @afharo

@afharo afharo merged commit 52ab19d into elastic:main Feb 25, 2025
11 checks passed
@afharo afharo deleted the es-9.0.0-alpha.2 branch February 25, 2025 14:37
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 9.0

https://github.com/elastic/kibana/actions/runs/13523771765

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
9.0 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 208776

Questions ?

Please refer to the Backport tool documentation

@afharo afharo removed the backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) label Feb 25, 2025
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Feb 25, 2025
@afharo
Copy link
Member Author

afharo commented Feb 25, 2025

Skipping backport since the amount of conflicts in the 9.0 branch would create too many risks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting ci:build-serverless-image ci:project-deploy-observability Create an Observability project elasticsearch-js-9 Feature:elasticsearch :ml release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team:Detection Engine Security Solution Detection Engine Area Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team:Entity Analytics Security Entity Analytics Team Team:ESQL ES|QL related features in Kibana Team:Fleet Team label for Observability Data Collection Fleet team Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more Team:Logstash Team:ML Team label for ML (also use :ml) Team:Monitoring Stack Monitoring team Team:Obs AI Assistant Observability AI Assistant Team:obs-entities Observability Entities Team Team:obs-knowledge Observability Experience Knowledge team Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team Team:obs-ux-logs Observability Logs User Experience Team Team:obs-ux-management Observability Management User Experience Team Team:Observability Team label for Observability Team (for things that are handled across all of observability) Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas Team:QA Team label for QA Team Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Team:Search Team:Security Generative AI Security Generative AI Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) Team:Threat Hunting:Explore Team:Threat Hunting:Investigations Security Solution Investigations Team Team:Threat Hunting Security Solution Threat Hunting Team Team:Visualizations Visualization editors, elastic-charts and infrastructure v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove deprecated 'body' key from Request interfaces