[HTTP] Log deprecated api usages #207904

jesuswr · 2025-01-22T19:14:29Z

Summary

Resolves #199616

Added a debug logger when calling deprecated APIs. This logger is disabled by default, to enable it you have to set deprecation.enable_http_debug_logs: true.

To test this you can play with the config value and do a request like this to a deprecated endpoint:

#!/bin/bash
API_KEY=""
curl -X GET "http://localhost:5601/api/cases/status" -i \
-H "Content-Type: application/json" \
-H "Authorization: ApiKey $API_KEY"

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

jesuswr · 2025-01-22T19:43:10Z

@elasticmachine merge upstream

jesuswr · 2025-01-22T19:43:21Z

/ci

jesuswr · 2025-01-23T08:58:27Z

/ci

jesuswr · 2025-01-23T16:48:57Z

/ci

jesuswr · 2025-01-24T11:59:57Z

@elasticmachine merge upstream

jesuswr · 2025-01-24T14:48:37Z

/ci

florent-leborgne

Thanks for adding docs! ☀️ I left a few suggestions.

docs/settings/logging-settings.asciidoc

docs/setup/upgrade.asciidoc

docs/settings/logging-settings.asciidoc

Co-authored-by: florent-leborgne <[email protected]>

elasticmachine · 2025-01-27T09:30:46Z

Pinging @elastic/kibana-core (Team:Core)

jesuswr · 2025-01-27T09:31:28Z

@elasticmachine merge upstream

jesuswr · 2025-01-27T09:31:32Z

/ci

Already reviewed

jesuswr · 2025-01-27T14:50:48Z

Not sure what label should be used for release notes and backport in this PR 🤔

jloleysens

Overall LGTM! Great work @jesuswr. Docs additions are also really nice! Would be nice to avoid logging/building ECS log with the isLogLevelEnabled check. Let me know what you think!

Left mostly nit level comments. Would also be nice to update the description with the latest config name 👍🏻

Tested locally with:

deprecation.allow_http_debug_logs: true
logging:
  appenders:
    json-layout:
      type: console
      layout:
        type: json
  root:
    appenders: [json-layout]

Got log:

{
  "client": {
    "ip": "127.0.0.1"
  },
  "http": {
    "request": {
      "method": "GET",
      "mime_type": null,
      "referrer": "",
      "headers": {
        "host": "localhost:5601",
        "authorization": "[REDACTED]",
        "user-agent": "curl/8.7.1",
        "accept": "*/*",
        "content-type": "application/json"
      }
    }
  },
  "url": {
    "path": "/api/cases/status",
    "query": ""
  },
  "user_agent": {
    "original": "curl/8.7.1"
  },
  "trace": {
    "id": "29d8effd10396f026ddcc610fe1b9e41"
  },
  "service": {
    "node": {
      "roles": [
        "background_tasks",
        "ui"
      ]
    }
  },
  "ecs": {
    "version": "8.11.0"
  },
  "@timestamp": "2025-01-28T12:20:42.223+01:00",
  "message": "GET /api/cases/status",
  "log": {
    "level": "DEBUG",
    "logger": "deprecations-service.http"
  },
  "process": {
    "pid": 16753,
    "uptime": 32.599992709
  },
  "transaction": {
    "id": "a147b4d8aae01cad"
  }
}

src/core/packages/deprecations/server-internal/src/deprecation_config.ts

src/core/packages/deprecations/server-internal/src/routes/post_validation_handler.ts

src/core/packages/http/server-internal/src/logging/get_response_log.ts

jloleysens · 2025-01-28T11:10:55Z

src/core/packages/http/server-internal/src/logging/get_response_log.ts

  const bytesMsg = bytes ? ` - ${numeral(bytes).format('0.0b')}` : '';

  const traceId = (request.app as KibanaRequestState).traceId;

+  const responseLogObj = response


Good job making this more robust! Is it possible that we can add some test coverage in src/core/packages/http/server-internal/src/logging/get_response_log.test.ts?

src/core/packages/root/server-internal/src/server.ts

…_config.ts Co-authored-by: Jean-Louis Leysens <[email protected]>

…_validation_handler.ts Co-authored-by: Jean-Louis Leysens <[email protected]>

…e_log.ts Co-authored-by: Jean-Louis Leysens <[email protected]>

elasticmachine · 2025-01-28T14:14:45Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: c356682

Failed CI Steps

FTR Configs #36

Test Failures

[job] [logs] FTR Configs #36 / Rules Management - Prebuilt Rules - Prebuilt Rules Management @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules should install fleet packages required for detection engine to function

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`@kbn/core-http-server-internal`	2	1	-1

Total ESLint disabled count

id	before	after	diff
`@kbn/core-http-server-internal`	2	1	-1

History

💔 Build #270913 failed bf91b1e
💚 Build #270361 succeeded 13c6a19
💚 Build #270062 succeeded 7dee6a8
💔 Build #269774 failed f3a5dd1
💚 Build #269549 succeeded 55f1fa2

cc @jesuswr

kibanamachine · 2025-01-28T14:28:33Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/13012637634

## Summary Resolves elastic#199616 Added a debug logger when calling deprecated APIs. This logger is disabled by default, to enable it you have to set `deprecation.enable_http_debug_logs: true`. To test this you can play with the config value and do a request like this to a deprecated endpoint: ```shell #!/bin/bash API_KEY="" curl -X GET "http://localhost:5601/api/cases/status" -i \ -H "Content-Type: application/json" \ -H "Authorization: ApiKey $API_KEY" ``` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. --------- Co-authored-by: Elastic Machine <[email protected]> Co-authored-by: kibanamachine <[email protected]> Co-authored-by: florent-leborgne <[email protected]> Co-authored-by: Jean-Louis Leysens <[email protected]> (cherry picked from commit 23b7f0f)

kibanamachine · 2025-01-28T14:33:54Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.x`: - [[HTTP] Log deprecated api usages (#207904)](#207904)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Jesus Wahrman <[email protected]>

jesuswr added 3 commits January 22, 2025 15:24

add logger with random message

1c09922

add config, need to improve log message

449b082

add correct format to debug log

0a05840

jesuswr added release_note:enhancement Feature:http Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc deprecation_warnings labels Jan 22, 2025

jesuswr self-assigned this Jan 22, 2025

jesuswr added 2 commits January 22, 2025 20:15

import only types

6694076

fixed tests, now only need to add new tests

a68f80d

Merge branch 'main' into log-deprecated-api-usages

9dbefef

[CI] Auto-commit changed files from 'node scripts/notice'

55f1fa2

jesuswr added 5 commits January 23, 2025 13:08

update func

3432da2

tests for get ecs response log

6d22cd0

rename config value

9ca7227

add test for setup

3676126

add tests for new config value

f3a5dd1

elasticmachine and others added 2 commits January 24, 2025 11:59

Merge branch 'main' into log-deprecated-api-usages

9bb442d

add documentation for new config value

7dee6a8

florent-leborgne self-requested a review January 24, 2025 15:34

florent-leborgne previously requested changes Jan 27, 2025

View reviewed changes

docs/settings/logging-settings.asciidoc Outdated Show resolved Hide resolved

docs/setup/upgrade.asciidoc Outdated Show resolved Hide resolved

docs/setup/upgrade.asciidoc Outdated Show resolved Hide resolved

florent-leborgne reviewed Jan 27, 2025

View reviewed changes

docs/settings/logging-settings.asciidoc Outdated Show resolved Hide resolved

jesuswr and others added 2 commits January 27, 2025 09:58

Update docs/settings/logging-settings.asciidoc

187cd95

Co-authored-by: florent-leborgne <[email protected]>

Update docs/setup/upgrade.asciidoc

cfda527

Co-authored-by: florent-leborgne <[email protected]>

jesuswr added the backport label Jan 27, 2025

Merge branch 'main' into log-deprecated-api-usages

13c6a19

jesuswr added backport:skip This commit does not require backporting and removed backport labels Jan 27, 2025

jesuswr requested a review from jloleysens January 28, 2025 10:00

jloleysens approved these changes Jan 28, 2025

View reviewed changes

jesuswr and others added 6 commits January 28, 2025 12:29

Update src/core/packages/deprecations/server-internal/src/deprecation…

53d24f4

…_config.ts Co-authored-by: Jean-Louis Leysens <[email protected]>

Update src/core/packages/deprecations/server-internal/src/routes/post…

ab09f10

…_validation_handler.ts Co-authored-by: Jean-Louis Leysens <[email protected]>

Update src/core/packages/http/server-internal/src/logging/get_respons…

bf91b1e

…e_log.ts Co-authored-by: Jean-Louis Leysens <[email protected]>

rename config

e5a91d3

fix reference in docs

3963a03

add more tests to get ecs response log

c356682

jesuswr added backport:version Backport to applied version labels v8.18.0 and removed backport:skip This commit does not require backporting labels Jan 28, 2025

jesuswr merged commit 23b7f0f into elastic:main Jan 28, 2025
13 checks passed

kibanamachine added the v9.0.0 label Jan 28, 2025

jesuswr deleted the log-deprecated-api-usages branch January 28, 2025 14:32

kibanamachine mentioned this pull request Jan 28, 2025

[8.x] [HTTP] Log deprecated api usages (#207904) #208548

Merged

jloleysens mentioned this pull request Jan 28, 2025

[UA] Link to docs about HTTP API deprecation logger #208570

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HTTP] Log deprecated api usages #207904

[HTTP] Log deprecated api usages #207904

jesuswr commented Jan 22, 2025 •

edited by kibanamachine

Loading

jesuswr commented Jan 22, 2025

jesuswr commented Jan 22, 2025

jesuswr commented Jan 23, 2025

jesuswr commented Jan 23, 2025

jesuswr commented Jan 24, 2025

jesuswr commented Jan 24, 2025

florent-leborgne left a comment •

edited

Loading

elasticmachine commented Jan 27, 2025

jesuswr commented Jan 27, 2025

jesuswr commented Jan 27, 2025

jesuswr commented Jan 27, 2025

jloleysens left a comment •

edited

Loading

jloleysens Jan 28, 2025

jesuswr Jan 28, 2025

elasticmachine commented Jan 28, 2025

ESLint disabled line counts

Total ESLint disabled count

kibanamachine commented Jan 28, 2025

kibanamachine commented Jan 28, 2025

[HTTP] Log deprecated api usages #207904

[HTTP] Log deprecated api usages #207904

Conversation

jesuswr commented Jan 22, 2025 • edited by kibanamachine Loading

Summary

Checklist

Identify risks

jesuswr commented Jan 22, 2025

jesuswr commented Jan 22, 2025

jesuswr commented Jan 23, 2025

jesuswr commented Jan 23, 2025

jesuswr commented Jan 24, 2025

jesuswr commented Jan 24, 2025

florent-leborgne left a comment • edited Loading

Choose a reason for hiding this comment

elasticmachine commented Jan 27, 2025

jesuswr commented Jan 27, 2025

jesuswr commented Jan 27, 2025

jesuswr commented Jan 27, 2025

jloleysens left a comment • edited Loading

Choose a reason for hiding this comment

jloleysens Jan 28, 2025

Choose a reason for hiding this comment

jesuswr Jan 28, 2025

Choose a reason for hiding this comment

elasticmachine commented Jan 28, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

ESLint disabled line counts

Total ESLint disabled count

History

kibanamachine commented Jan 28, 2025

kibanamachine commented Jan 28, 2025

💚 All backports created successfully

Questions ?

jesuswr commented Jan 22, 2025 •

edited by kibanamachine

Loading

florent-leborgne left a comment •

edited

Loading

jloleysens left a comment •

edited

Loading