-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] Unordered consuming case in Key_Shared subscription #12885
Labels
Milestone
Comments
Jason918
pushed a commit
to Jason918/pulsar
that referenced
this issue
Nov 19, 2021
Jason918
pushed a commit
to Jason918/pulsar
that referenced
this issue
Nov 19, 2021
2 tasks
@Jason918 - good catch! Do you plan to submit a patch for this bug? |
@michaeljmarshall Already have a patch here #12890 |
@codelipenghui - oh, right. I don't know how I missed the GitHub note right about my comment. Thanks! |
Jason918
added a commit
to Jason918/pulsar
that referenced
this issue
Nov 21, 2021
Jason918
added a commit
to Jason918/pulsar
that referenced
this issue
Nov 26, 2021
Jason918
added a commit
to Jason918/pulsar
that referenced
this issue
Nov 30, 2021
Jason918
added a commit
to Jason918/pulsar
that referenced
this issue
Nov 30, 2021
codelipenghui
pushed a commit
that referenced
this issue
Nov 30, 2021
zeo1995
pushed a commit
to zeo1995/pulsar
that referenced
this issue
Dec 1, 2021
* up/master: (75 commits) [website][upgrade]feat: website upgrade / docs migration - 2.5.1 Get Started/Concepts and Architecture/Pulsar Schema (apache#13030) Fix environment variable assignment in startup scripts (apache#13025) update 2.8.x (apache#13029) [Doc] add tips for Pulsar tools (apache#13044) Suggest to use tlsPort instead of deprecated TlsEnable (apache#13039) Integration tests for function-worker rebalance and drain operations. (apache#13058) fix(functions): missing runtime set in GoInstanceConfig (apache#13031) [pulsar-admin] Add get-replicated-subscription-status command for topic (apache#12891) [Broker] Consider topics in pulsar/system namespace as system topics (apache#13050) Fix typo: correct sizeUint to sizeUnit (apache#13040) fix-12894 (apache#12896) Don't attempt to delete pending ack store unless transactions are enabled (apache#13041) [Perf] Evaluate the current protocol version once (apache#13045) Fix Issue apache#12885, Unordered consuming case in Key_Shared subscription (apache#12890) [broker]Optimize topicMaxMessageSize with topic local cache. (apache#12830) [PIP-105] Part-2 Support pluggable entry filter in Dispatcher (apache#12970) [website] Modify admin-api-topic.md document (apache#12996) add missed import (apache#13037) [metadata] Add RocksdbMetadataStore (apache#12776) [C] Add pulsar_client_subscribe_multi_topics and pulsar_client_subscribe_pattern (apache#12965) ... # Conflicts: # site2/website-next/docusaurus.config.js # site2/website-next/versioned_sidebars/version-2.6.1-sidebars.json # site2/website-next/versions.json
eolivelli
pushed a commit
that referenced
this issue
Dec 15, 2021
fxbing
pushed a commit
to fxbing/pulsar
that referenced
this issue
Dec 19, 2021
zymap
pushed a commit
that referenced
this issue
Dec 23, 2021
eolivelli
pushed a commit
to datastax/pulsar
that referenced
this issue
Feb 3, 2022
…iption (apache#12890) (cherry picked from commit 73ef162)
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Describe the bug
It's a previous flaky test in
org.apache.pulsar.client.api.KeySharedSubscriptionTest#testRemoveFirstConsumer
See #11426
There is a race condition in
PersistentDispatcherMultipleConsumers
when previous client closed with unack messaages.Detailed Explanation:
Here is the core steps of this case.
The race condition happens after step6.
Just before step6, there is a retry loop going on for this consumer.
After we close c1, PersistentDispatcherMultipleConsumers#removeConsumer will be called like this.
If Thread3 happens after Thread2, everything is ok, and this is most of the real cases. As time between Thread1 and Thread2 is quite small.
But if it does happens in the order of Thread1 --> Thread3 --> Thread2. The disorder occurs.
In the
readMoreEntries
of 3.3 , calculateToRead return -1,-1 because havePendingReplayRead is true in 1.2 of Thread1. So thereadMoreEntries
will just returns without further actions.Then in Thread2, previous reading ended, and we got m10-m14 to dispatch. now we can select consumer c2 because we have no more c1 for waiting. So the client c2 got m10 instead of m0.
To Reproduce
Steps to reproduce the behavior:
pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java
Line 294 in 5523604
org.apache.pulsar.client.api.KeySharedSubscriptionTest#testRemoveFirstConsumer
. It will hit the race condition in a few retries.Expected behavior
As the case designed in
testRemoveFirstConsumer
. The consuming order should be right.Screenshots
Nop
Desktop (please complete the following information):
Additional context
NO
The text was updated successfully, but these errors were encountered: