[Bug] In the case of single-replica synchronous disk flushing, messages may be lost during a crash

[RongtongJin]

Before Creating the Bug Report

• I found a bug, not just asking a question, which should be created in GitHub Discussions.

• I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.

• I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.

Runtime platform environment

All platforms

RocketMQ version

All versions

JDK Version

Not related to the JDK version

Describe the Bug

As shown in the image, the consumer might read unflushed messages, leading to premature consumption and a forward shift of the consumption offset. After a crash, the consumeQueue may roll back, and new messages could have a chance of not being consumed.

Steps to Reproduce

In a single-replica scenario with synchronous disk flushing configured, send a message and perform a crash test to check for message loss.

What Did You Expect to See?

No Messages lost.

What Did You See Instead?

Messages lost.

Additional Context

Under the condition of synchronous disk flushing, unflushed messages should be unacknowledged messages. Such messages should not be consumed prematurely, as this would lead to the consumption offset moving forward, resulting in potential phantom reads or message loss on the consumer side.