Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #8239] Fix the issue of potential message loss after a crash under synchronous disk flushing configuration. #8240

Merged
merged 4 commits into from
Jun 7, 2024

Conversation

RongtongJin
Copy link
Contributor

Which Issue(s) This PR Fixes

Fixes #8239

Brief Description

Under the condition of synchronous disk flushing, unflushed messages should be unacknowledged messages. Such messages should not be consumed prematurely, as this would lead to the consumption offset moving forward, resulting in potential phantom reads and message loss on the consumer side.

Therefore, in this PR, the dispatch offset of the ConsumeQueue is restricted to be before the disk flush offset to prevent consumers from experiencing phantom reads.

How Did You Test This Change?

Add a UT

@codecov-commenter
Copy link

codecov-commenter commented Jun 1, 2024

Codecov Report

Attention: Patch coverage is 40.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 42.89%. Comparing base (7a5ea90) to head (4dfcd53).
Report is 9 commits behind head on develop.

Files Patch % Lines
...org/apache/rocketmq/store/DefaultMessageStore.java 20.00% 1 Missing and 3 partials ⚠️
...ache/rocketmq/store/config/MessageStoreConfig.java 50.00% 2 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #8240      +/-   ##
=============================================
- Coverage      42.89%   42.89%   -0.01%     
- Complexity     10368    10381      +13     
=============================================
  Files           1270     1271       +1     
  Lines          88724    88827     +103     
  Branches       11406    11420      +14     
=============================================
+ Hits           38060    38103      +43     
- Misses         45956    46015      +59     
- Partials        4708     4709       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

yuz10
yuz10 previously approved these changes Jun 5, 2024
lizhimins
lizhimins previously approved these changes Jun 6, 2024
@RongtongJin RongtongJin dismissed stale reviews from lizhimins and yuz10 via b938258 June 6, 2024 10:01
Copy link
Contributor

@guyinyou guyinyou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@lizhimins lizhimins merged commit d60198f into apache:develop Jun 7, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] In the case of single-replica synchronous disk flushing, messages may be lost during a crash
6 participants