Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] Fix system service namespace create internal event topic. #17867

Merged
merged 5 commits into from
Oct 13, 2022

Conversation

Technoboy-
Copy link
Contributor

Motivation

Fix system service namespace(heartbeat namespace) creates internal __change_events topics:

2022-09-28T17:04:24,833 - INFO  - [metadata-store-12-1:ServerCnx@1474] - [/127.0.0.1:50223] Created new producer: Producer{topic=SystemTopic{topic=persistent://pulsar/localhost:50220/healthcheck}, client=/127.0.0.1:50223, producerName=test-0-0, producerId=0}
2022-09-28T17:04:24,845 - INFO  - [pulsar-io-6-3:ConsumerImpl@822] - [persistent://pulsar/localhost:50220/__change_events-partition-0][multiTopicsReader-895a02f0d4] Subscribing to topic on cnx [id: 0xc6ef1b83, L:/127.0.0.1:50223 - R:localhost/127.0.0.1:50218], consumerId 0
2022-09-28T17:04:24,848 - INFO  - [pulsar-io-6-3:ConsumerImpl@822] - [persistent://pulsar/localhost:50220/__change_events-partition-1][multiTopicsReader-895a02f0d4] Subscribing to topic on cnx [id: 0xc6ef1b83, L:/127.0.0.1:50223 - R:localhost/127.0.0.1:50218], consumerId 1
2022-09-28T17:04:24,849 - INFO  - [pulsar-io-6-3:ConsumerImpl@822] - [persistent://pulsar/localhost:50220/__change_events-partition-2][multiTopicsReader-895a02f0d4] Subscribing to topic on cnx [id: 0xc6ef1b83, L:/127.0.0.1:50223 - R:localhost/127.0.0.1:50218], consumerId 2
2022-09-28T17:04:24,853 - INFO  - [pulsar-io-6-3:ConsumerImpl@822] - [persistent://pulsar/localhost:50220/__change_events-partition-3][multiTopicsReader-895a02f0d4] Subscribing to topic on cnx [id: 0xc6ef1b83, L:/127.0.0.1:50223 - R:localhost/127.0.0.1:50218], consumerId 3
2022-09-28T17:04:24,853 - INFO  - [pulsar-io-6-3:ConsumerImpl@822] - [persistent://pulsar/localhost:50220/__change_events-partition-4][multiTopicsReader-895a02f0d4] Subscribing to topic on cnx [id: 0xc6ef1b83, L:/127.0.0.1:50223 - R:localhost/127.0.0.1:50218], consumerId 4
2022-09-28T17:04:24,856 - INFO  - [pulsar-io-6-3:ProducerImpl@1694] - [persistent://pulsar/localhost:50220/healthcheck] [test-0-0] Created producer on cnx [id: 0xc6ef1b83, L:/127.0.0.1:50223 - R:localhost/127.0.0.1:50218]

Documentation

  • doc-not-needed
    (Please explain why)

Matching PR in forked repository

PR in forked repository: https://github.com/Technoboy-/pulsar/pull/8

@Technoboy- Technoboy- self-assigned this Sep 28, 2022
@Technoboy- Technoboy- added this to the 2.12.0 milestone Sep 28, 2022
@Technoboy- Technoboy- added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/broker labels Sep 28, 2022
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 28, 2022
@@ -1567,7 +1567,8 @@ public CompletableFuture<ManagedLedgerConfig> getManagedLedgerConfig(TopicName t
RetentionPolicies retentionPolicies = null;
OffloadPoliciesImpl topicLevelOffloadPolicies = null;

if (pulsar.getConfig().isTopicLevelPoliciesEnabled()) {
if (pulsar.getConfig().isTopicLevelPoliciesEnabled()
&& !NamespaceService.isSystemServiceNamespace(namespace.toString())) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can also move this check !NamespaceService.isSystemServiceNamespace(namespace.toString()) to pulsar.getTopicPoliciesService().getTopicPolicies(topicName)?

Because I see you already check it at SystemTopicBasedTopicPoliciesService#sendTopicPolicyEvent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, we should avoid creating the topic first. This is the root cause.

@@ -101,6 +101,10 @@ public CompletableFuture<Void> updateTopicPoliciesAsync(TopicName topicName, Top

private CompletableFuture<Void> sendTopicPolicyEvent(TopicName topicName, ActionType actionType,
TopicPolicies policies) {
if (NamespaceService.isHeartbeatNamespace(topicName.getNamespaceObject())) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we use NamespaceService#isSystemServiceNamespace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the same with line 240.

@Technoboy- Technoboy- force-pushed the fix-health-check-topic-policy-1 branch from ee9c33b to e5c3372 Compare October 10, 2022 04:37
@Technoboy- Technoboy- force-pushed the fix-health-check-topic-policy-1 branch from e5c3372 to cf1d485 Compare October 12, 2022 07:03
@codecov-commenter
Copy link

codecov-commenter commented Oct 13, 2022

Codecov Report

❗ No coverage uploaded for pull request base (master@7e420c6). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master   #17867   +/-   ##
=========================================
  Coverage          ?   46.73%           
  Complexity        ?    17838           
=========================================
  Files             ?     1573           
  Lines             ?   128214           
  Branches          ?    14099           
=========================================
  Hits              ?    59923           
  Misses            ?    62135           
  Partials          ?     6156           
Flag Coverage Δ
unittests 46.73% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

@Technoboy- Technoboy- merged commit 29baa0b into apache:master Oct 13, 2022
@Technoboy- Technoboy- modified the milestones: 2.12.0, 2.11.0 Oct 13, 2022
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Oct 13, 2022
liangyepianzhou pushed a commit that referenced this pull request Dec 12, 2022
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jan 10, 2023
…c. (apache#17867)

(cherry picked from commit 29baa0b)
(cherry picked from commit 14d68a4)
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jan 11, 2023
…c. (apache#17867)

(cherry picked from commit 29baa0b)
(cherry picked from commit 14d68a4)
@Technoboy- Technoboy- deleted the fix-health-check-topic-policy-1 branch November 11, 2023 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/broker cherry-picked/branch-2.10 cherry-picked/branch-2.11 doc-not-needed Your PR changes do not impact docs ready-to-test release/2.10.3 type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants