Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: BatchQuery executor read with expired epoch under parallel e2e #5592

Closed
BugenZhao opened this issue Sep 27, 2022 · 0 comments · Fixed by #5606
Closed

bug: BatchQuery executor read with expired epoch under parallel e2e #5592

BugenZhao opened this issue Sep 27, 2022 · 0 comments · Fixed by #5606
Assignees
Labels
component/meta Meta related issue. component/streaming Stream processing related issue. type/bug Something isn't working

Comments

@BugenZhao
Copy link
Member

Recently we've enabled the watermark checking, and some bugs of reading with expired epochs are revealed. The batch query executor may read with an expired epoch under parallel e2e, which might be caused by the epoch of creating mview failing to be pinned.

https://buildkite.com/risingwavelabs/pull-request/builds/9338#01837ead-70bb-4053-8d84-d9ef51e23c5f

The unpin here might be problematic. With concurrent DDL, a calling of unpin may also unpin the snapshot for other creating mviews.

// TODO: refactor this
if is_create_mv {
// The snapshot ingestion may last for several epochs, we should pin the epoch here.
// TODO: this should be done in `post_collect`
let _snapshot = self.hummock_manager.pin_snapshot(META_NODE_ID).await?;
// Wait for this command to be finished.
let res = finish_rx.await;
self.hummock_manager.unpin_snapshot(META_NODE_ID).await?;
res.map_err(|e| anyhow!("failed to finish command: {}", e))?;
} else {
// Wait for this command to be finished.
finish_rx
.await
.map_err(|e| anyhow!("failed to finish command: {}", e))?;
}

@BugenZhao BugenZhao added type/bug Something isn't working component/streaming Stream processing related issue. component/meta Meta related issue. labels Sep 27, 2022
@BugenZhao BugenZhao self-assigned this Sep 27, 2022
@github-actions github-actions bot added this to the release-0.1.14 milestone Sep 27, 2022
@mergify mergify bot closed this as completed in #5606 Sep 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/meta Meta related issue. component/streaming Stream processing related issue. type/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant