Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Race Condition inside PulsarClient at Datanode #7770

Closed
xiaofan-luan opened this issue Sep 12, 2021 · 1 comment · Fixed by #7909
Closed

Race Condition inside PulsarClient at Datanode #7770

xiaofan-luan opened this issue Sep 12, 2021 · 1 comment · Fixed by #7909
Assignees
@XuanYang-cn
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone
2.0.0-RC7

Comments

@xiaofan-luan
Copy link
Collaborator

Steps/Code to reproduce:

--- FAIL: TestDataNodeTtChannel/test_expire_allocation_after_receiving_tt_msg (0.20s)
    testing.go:1042: race detected during execution of test
testing.go:1042: race detected during execution of test

Expected result:

 The test should be passed without any failures.

Actual results:

Detailed Race information is shown here:
WARNING: DATA RACE
Read at 0x00c003795ca8 by goroutine 908:
github.com/apache/pulsar-client-go/pulsar.(*partitionConsumer).internalClose()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_partition.go:854 +0x687
github.com/apache/pulsar-client-go/pulsar.(*partitionConsumer).runEventsLoop()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_partition.go:808 +0x29b

Previous write at 0x00c003795ca8 by goroutine 909:
github.com/apache/pulsar-client-go/pulsar.(*partitionConsumer).grabConn()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_partition.go:970 +0x17c6
github.com/apache/pulsar-client-go/pulsar.(*partitionConsumer).reconnectToBroker()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_partition.go:883 +0x238
github.com/apache/pulsar-client-go/pulsar.(*partitionConsumer).runEventsLoop.func2()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_partition.go:787 +0xab

Goroutine 908 (running) created at:
github.com/apache/pulsar-client-go/pulsar.newPartitionConsumer()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_partition.go:208 +0xf3a
github.com/apache/pulsar-client-go/pulsar.(*consumer).internalTopicSubscribeToPartitions.func1()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_impl.go:326 +0x778

Goroutine 909 (running) created at:
github.com/apache/pulsar-client-go/pulsar.(*partitionConsumer).runEventsLoop()
/go/pkg/mod/github.com/apache/[email protected]/pulsar/consumer_partition.go:780 +0x146

Environment:

  • Milvus version(e.g. v2.0.0-RC2 or 8b23a93):
  • Deployment mode(standalone or cluster):
  • SDK version(e.g. pymilvus v2.0.0rc2):
  • OS(Ubuntu or CentOS):
  • CPU/Memory:
  • GPU:
  • Others:
@xiaofan-luan xiaofan-luan added kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on. labels Sep 12, 2021
@xiaofan-luan xiaofan-luan added this to the 2.0.0-RC7 milestone Sep 12, 2021
@XuanYang-cn XuanYang-cn self-assigned this Sep 12, 2021
@XuanYang-cn
Copy link
Contributor

There's a issue here apache/pulsar-client-go#580

It seemed to be fixed on pulsar-client-go 0.6.0, maybe we should considering upgrade our pulsar-client-go to 0.6.0

apache/pulsar-client-go#535

@XuanYang-cn XuanYang-cn mentioned this issue Sep 14, 2021
Merged
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Sep 15, 2021
@XuanYang-cn
Upgrade pulsar-client-go to 0.6.0
b410f39 
MessageID interface of pulsar-client-go adds
4 more methods. So I add these methods into
`MessageID` interface.

Resolves: milvus-io#7770

Signed-off-by: yangxuan <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Sep 15, 2021
@XuanYang-cn
Upgrade pulsar-client-go to 0.6.0 (#7909)
ff8fef6 
MessageID interface of pulsar-client-go adds
4 more methods. So I add these methods into
`MessageID` interface.

Resolves: #7770

Signed-off-by: yangxuan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@XuanYang-cn XuanYang-cn

Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
2.0.0-RC7
Development

Successfully merging a pull request may close this issue.

Upgrade pulsar-client-go to 0.6.0 XuanYang-cn/milvus
2 participants
@XuanYang-cn @xiaofan-luan