Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix hummock snapshot release in batch local execution #5971

Merged
merged 2 commits into from
Oct 23, 2022

Conversation

hzxa21
Copy link
Collaborator

@hzxa21 hzxa21 commented Oct 22, 2022

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

Prior to this PR, batch query in local execution mode will release the hummock snapshot before query execution, which can cause the read epoch to be expired when reading from state store during query execution.

This PR fixes the issue by releasing the hummock snapshot after query execution is completed. Note the this is different from what we did for batch query in distributed execution mode, in which the hummock snapshot is released immediately after all scan executors are scheduled before the completion of query execution. I think this is acceptable since batch query in local execution mode is supposed to be fast and shouldn't hold the hummock snapshot for too long.

Checklist

  • I have written necessary rustdoc comments
    - [ ] I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

Fix #5711

@github-actions github-actions bot added the type/fix Bug fix label Oct 22, 2022
@codecov
Copy link

codecov bot commented Oct 22, 2022

Codecov Report

Merging #5971 (cd4412b) into main (5ffeccb) will decrease coverage by 0.00%.
The diff coverage is 31.25%.

@@            Coverage Diff             @@
##             main    #5971      +/-   ##
==========================================
- Coverage   74.58%   74.58%   -0.01%     
==========================================
  Files         924      924              
  Lines      147411   147420       +9     
==========================================
+ Hits       109943   109947       +4     
- Misses      37468    37473       +5     
Flag Coverage Δ
rust 74.58% <31.25%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/frontend/src/handler/query.rs 20.24% <0.00%> (ø)
src/frontend/src/scheduler/local.rs 0.00% <0.00%> (ø)
...frontend/src/scheduler/hummock_snapshot_manager.rs 61.35% <50.00%> (-0.34%) ⬇️
src/frontend/src/scheduler/distributed/query.rs 74.87% <100.00%> (+0.06%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! LGTM

@mergify mergify bot merged commit d72ba12 into main Oct 23, 2022
@mergify mergify bot deleted the patrick/fix-5711 branch October 23, 2022 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flaky test: Failed to build executor due to Expired Epoch
3 participants