Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find blocks in store instead of in chain #1324

Merged

Conversation

area363
Copy link
Member

@area363 area363 commented Jun 2, 2021

This PR changes the BlockCompletion predicate to use the ContainsBlock method of IStore instead of BlockChain. This way, Swarm won't have to download blocks again if preloading is aborted during the Block Execution stage.

@area363 area363 requested review from dahlia and a team June 2, 2021 04:50
@area363 area363 self-assigned this Jun 2, 2021
@area363 area363 requested review from kfangw and limebell and removed request for a team June 2, 2021 04:50
Copy link
Contributor

@dahlia dahlia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a minimum test case to reproduce the problem this PR tries to address? Would it be difficult to reproduce in deterministic manner?

@longfin
Copy link
Member

longfin commented Jun 3, 2021

it would be better if we have changelog about this patch. (as behaviour change?)

@dahlia dahlia changed the title [skip changelog] find blocks in store instead of in chain Find blocks in store instead of in chain Jun 7, 2021
@stale
Copy link

stale bot commented Jun 22, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale The issue is stale label Jun 22, 2021
@stale
Copy link

stale bot commented Jun 30, 2021

This pull request has been automatically closed because it has not had recent activity. Thank you for your contributions.

@stale stale bot closed this Jun 30, 2021
@longfin longfin reopened this Jul 5, 2021
@stale stale bot removed the stale The issue is stale label Jul 5, 2021
@limebell limebell requested a review from dahlia July 6, 2021 05:09
@limebell limebell force-pushed the fix/prevent-block-redownload branch 2 times, most recently from 1073ca5 to 730d7d8 Compare July 6, 2021 09:51
@codecov
Copy link

codecov bot commented Jul 6, 2021

Codecov Report

Merging #1324 (d1e40d3) into 0.12-maintenance (40e7735) will decrease coverage by 0.24%.
The diff coverage is 100.00%.

❗ Current head d1e40d3 differs from pull request most recent head b590f34. Consider uploading reports for the commit b590f34 to get more accurate results

@@                 Coverage Diff                  @@
##           0.12-maintenance    #1324      +/-   ##
====================================================
- Coverage             77.34%   77.09%   -0.25%     
====================================================
  Files                   256      254       -2     
  Lines                 17229    17145      -84     
====================================================
- Hits                  13325    13218     -107     
- Misses                 3353     3371      +18     
- Partials                551      556       +5     
Impacted Files Coverage Δ
Libplanet/Net/Swarm.cs 84.82% <100.00%> (-0.05%) ⬇️
Libplanet/Store/StoreExtensions.cs 64.70% <0.00%> (-11.16%) ⬇️
Libplanet/Net/SwarmOptions.cs 92.30% <0.00%> (-7.70%) ⬇️
Libplanet/ByteUtil.cs 88.23% <0.00%> (-5.42%) ⬇️
Libplanet/Net/Protocols/RoutingTable.cs 87.89% <0.00%> (-4.86%) ⬇️
Libplanet/Blocks/Block.cs 86.55% <0.00%> (-1.64%) ⬇️
Libplanet/Blockchain/BlockChain.cs 90.51% <0.00%> (-1.16%) ⬇️
Libplanet/Store/BlockSet.cs 84.48% <0.00%> (-0.52%) ⬇️
Libplanet.RocksDBStore/MonoRocksDBStore.cs 90.63% <0.00%> (-0.41%) ⬇️
Libplanet/Blockchain/Policies/BlockPolicy.cs 91.76% <0.00%> (-0.38%) ⬇️
... and 19 more

@limebell limebell force-pushed the fix/prevent-block-redownload branch from 730d7d8 to baa6e85 Compare July 8, 2021 01:19
@limebell
Copy link
Member

limebell commented Jul 8, 2021

PTAL @libplanet

dahlia
dahlia previously approved these changes Jul 8, 2021
kfangw
kfangw previously approved these changes Jul 8, 2021
@kfangw
Copy link
Contributor

kfangw commented Jul 8, 2021

LGTM.
I am not sure, but if there is performance issue(db read is expensive than memory read) then, how about the following?

  • look up the blockchain first. If not exists, look up the store

@limebell
Copy link
Member

limebell commented Jul 8, 2021

LGTM.
I am not sure, but if there is performance issue(db read is expensive than memory read) then, how about the following?

  • look up the blockchain first. If not exists, look up the store

Looking up in the blockchain looks up storage too.

@kfangw
Copy link
Contributor

kfangw commented Jul 8, 2021

LGTM.
I am not sure, but if there is performance issue(db read is expensive than memory read) then, how about the following?

  • look up the blockchain first. If not exists, look up the store

Looking up in the blockchain looks up storage too.

  • BlockChain.ContainsBlock return true if blockchain and store both contains the block with other conditions
  • Store.ContainsBlock return true if store contains the block

IMO, what I mentioned(in blockchain but not in store) is not covered by BlockChain.ContainsBlock

Btw, BlockChain.ContainsBlock already searching db too, hence there is no further problem with performance by this change.

greymistcube
greymistcube previously approved these changes Jul 8, 2021
@greymistcube
Copy link
Contributor

I thought the main issue was unnecessarily redownloading a block where given block is not in blockchain but in store? Improving storage access performance is a separate issue.

@longfin
Copy link
Member

longfin commented Jul 19, 2021

/rebase

@libplanet libplanet dismissed stale reviews from greymistcube, kfangw, and dahlia via 32d44cd July 19, 2021 10:15
@libplanet libplanet force-pushed the fix/prevent-block-redownload branch from baa6e85 to 32d44cd Compare July 19, 2021 10:15
@dahlia
Copy link
Contributor

dahlia commented Jul 22, 2021

/rebase

@libplanet libplanet force-pushed the fix/prevent-block-redownload branch from 32d44cd to 8a93fb2 Compare July 22, 2021 01:02
dahlia
dahlia previously approved these changes Jul 22, 2021
@longfin
Copy link
Member

longfin commented Jul 26, 2021

/rebase

longfin
longfin previously approved these changes Jul 26, 2021
@libplanet libplanet dismissed stale reviews from longfin and dahlia via 13b05f4 July 26, 2021 08:46
@libplanet libplanet force-pushed the fix/prevent-block-redownload branch from 8a93fb2 to 13b05f4 Compare July 26, 2021 08:46
@dahlia dahlia changed the base branch from main to 0.12-maintenance July 26, 2021 08:46
@dahlia dahlia force-pushed the fix/prevent-block-redownload branch from 13b05f4 to b590f34 Compare July 26, 2021 13:38
@dahlia
Copy link
Contributor

dahlia commented Jul 26, 2021

Retargeted to the 0.12-maintenance branch and rebased on that.

@longfin longfin merged commit 63a9012 into planetarium:0.12-maintenance Jul 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants