Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure transactions_confirmed is idempotent #1861

Merged

Conversation

TheBlueMatt
Copy link
Collaborator

In many complexity-reduced implementations of chain syncing using
esplora transactions_confirmed may be called redundantly for
transactions which were already confirmed. To ensure this is
idempotent we add two new ConnectionStyles in our tests which
(a) call transactions_confirmed twice for each call, ensuring
simple idempotency is ensured and (b) call transactions_confirmed
once for each historical block every time we're connecting a new
block, ensuring we're fully idempotent even if every call is
repeated constantly.

In order to actually behave correctly this requires a simple
already-confirmed check in ChannelMonitor, which is included.

Example #53,637,290 of the old adage "that which isn't tested is broken".

In the next commit we'll add some checks that redundant
transactions aren't confirmed in different blocks, which would
cause test_htlc_ignore_latest_remote_commitment to fail. Here we
fix it to avoid the issue.
@TheBlueMatt TheBlueMatt added this to the 0.0.113 milestone Nov 18, 2022
@TheBlueMatt TheBlueMatt force-pushed the 2022-11-tx-connection-idempotency branch from a16d4ca to a005bf0 Compare November 18, 2022 19:16
In `ChannelMonitor`s, when a transaction containing a spend of a
revoked remote output reaches 6 confs, we may have no other
tracking of that txid remaining. Thus, if we see that transaction
again (because a user duplicatively confirms it), we'll generate a
redundant spendable output event for it.

Here we simply explicitly track all txids of transactions which
confirm with a spendable output, allowing us to check this
condition in the next commit.
@codecov-commenter
Copy link

codecov-commenter commented Nov 18, 2022

Codecov Report

Base: 90.61% // Head: 90.95% // Increases project coverage by +0.34% 🎉

Coverage data is based on head (a65a816) compared to base (7269fa2).
Patch coverage: 96.29% of modified lines in pull request are covered.

❗ Current head a65a816 differs from pull request most recent head cd315d5. Consider uploading reports for the commit cd315d5 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1861      +/-   ##
==========================================
+ Coverage   90.61%   90.95%   +0.34%     
==========================================
  Files          90       91       +1     
  Lines       47623    50270    +2647     
  Branches    47623    50270    +2647     
==========================================
+ Hits        43152    45723    +2571     
- Misses       4471     4547      +76     
Impacted Files Coverage Δ
lightning/src/ln/functional_tests.rs 97.02% <91.66%> (-0.14%) ⬇️
lightning/src/chain/channelmonitor.rs 90.94% <92.30%> (+0.24%) ⬆️
lightning/src/ln/functional_test_utils.rs 93.72% <100.00%> (+0.07%) ⬆️
lightning/src/ln/monitor_tests.rs 99.56% <100.00%> (+0.01%) ⬆️
lightning/src/ln/payment_tests.rs 98.77% <100.00%> (ø)
lightning/src/offers/offer.rs 92.26% <0.00%> (-2.30%) ⬇️
lightning-net-tokio/src/lib.rs 76.73% <0.00%> (-0.31%) ⬇️
lightning/src/offers/parse.rs 93.47% <0.00%> (ø)
lightning/src/ln/channelmanager.rs 85.08% <0.00%> (+0.04%) ⬆️
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@tnull tnull self-requested a review November 19, 2022 09:09
@TheBlueMatt TheBlueMatt force-pushed the 2022-11-tx-connection-idempotency branch from 315d120 to a65a816 Compare November 21, 2022 17:43
Copy link
Contributor

@tnull tnull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, would be ready for the squashing from my side.

Copy link
Contributor

@dunxen dunxen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also LGTM for squash.

In many complexity-reduced implementations of chain syncing using
esplora `transactions_confirmed` may be called redundantly for
transactions which were already confirmed. To ensure this is
idempotent we add two new `ConnectionStyle`s in our tests which
(a) call `transactions_confirmed` twice for each call, ensuring
simple idempotency is ensured and (b) call `transactions_confirmed`
once for each historical block every time we're connecting a new
block, ensuring we're fully idempotent even if every call is
repeated constantly.

In order to actually behave correctly this requires a simple
already-confirmed check in `ChannelMonitor`, which is included.
At the end of our `monitor_tests`, which test `ChannelMonitor`
`SpendableOutputs` and claimable `Balance`s, add new checks that
ensure that, if we're using the new
`ConnectStyle::HighlyRedundantTransactionsFirstSkippingBlocks`, we
can replay the full chain without getting redundant events or
`Balance`s.
@TheBlueMatt TheBlueMatt force-pushed the 2022-11-tx-connection-idempotency branch from a65a816 to cd315d5 Compare November 24, 2022 03:40
@TheBlueMatt
Copy link
Collaborator Author

Squashed without changes.

@TheBlueMatt TheBlueMatt merged commit 53eb0d7 into lightningdevkit:main Nov 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants