Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][monitor] Introduce new metrics related to transaction buffer snapshot segments #20125

Closed
wants to merge 4 commits into from
Closed

Conversation

liangyepianzhou
Copy link
Contributor

@liangyepianzhou liangyepianzhou commented Apr 18, 2023

Master #16913

Motivation

Enhance monitoring capabilities for the transaction buffer snapshot segments by introducing new metrics. These metrics help users better understand the performance and behavior of the transaction buffer, enabling them to make informed decisions for tuning and troubleshooting purposes.

Modification

Introduce new metrics related to transaction buffer snapshot segments, including counters for add, delete, and read operations, as well as gauges and histograms for snapshot indexes and index entry sizes. Additionally, update the AbortTxnProcessor to incorporate these new metrics and properly expose them through the Pulsar monitoring system.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: https://github.com/liangyepianzhou/pulsar/pull/31

Enhance monitoring capabilities for the transaction buffer snapshot segments by introducing new metrics. These metrics help users better understand the performance and behavior of the transaction buffer, enabling them to make informed decisions for tuning and troubleshooting purposes.

### Modification
Introduce new metrics related to transaction buffer snapshot segments, including counters for add, delete, and read operations, as well as gauges and histograms for snapshot indexes and index entry sizes. Additionally, update the `AbortTxnProcessor` to incorporate these new metrics and properly expose them through the Pulsar monitoring system.
@github-actions github-actions bot added doc-label-missing doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Apr 18, 2023
@apache apache deleted a comment from github-actions bot Apr 18, 2023
@github-actions github-actions bot added doc-required Your PR changes impact docs and you will update later. and removed doc-not-needed Your PR changes do not impact docs labels Apr 18, 2023
Copy link
Contributor

@congbobo184 congbobo184 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better implmentation like #16758

@github-actions
Copy link

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label May 19, 2023
@Technoboy- Technoboy- added this to the 3.2.0 milestone Jul 31, 2023
@liangyepianzhou liangyepianzhou closed this by deleting the head repository Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/metrics area/transaction doc-required Your PR changes impact docs and you will update later. Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants