[draft] [fix] [client] Could not consume messages after one partition deleted #21649
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Background:
Issue:
Regexp Topic & Partitioned Topics
: The Pulsar client stopped subscriptions of all partitions when one partition has been deleted, even if other partitions are still there. It causes the messages in other partitions not to be fully consumed.Regexp Topic
: The Pulsar client tries to create subscriptions for all partitions when the consumer starts, even if some partitions have been deleted. It causes the consumer not to start and throws an error "topic not found"( the issue of the current ticket )You can reproduce the issue by the test
testConsumerAfterOnePartDeleted
.Modifications
GetTopicsResult
into two classes:GetTopicsResult
: the fieldtopics
are the collection of the topic names that contains partition name. It is only used byClientCnx.GET_TOPICS_OF_NAMESPACE
.CollatedGetTopicsResult
: merge the topic names that are under the samePartitioned Topic
, It is only used in the client internal.TODO: Since this PR has changed a lot, the following things will be done in the subsequent PR:
MultiTopicsConsumerImpl
andPatternMultiTopicsConsumerImpl
TopicListWatcher
.Documentation
doc
doc-required
doc-not-needed
doc-complete
Matching PR in forked repository
PR in forked repository: x