chainntnfs: modify all historical rescans to scan backwards #2211
Conversation
|
We could also consider updating the height hint caches while doing rescans (not after the rescan is over), to let rescans pick up where they left when restarted. Not sure how well that would play with this change though. |
|
Yep we can follow up with that, would prefer to contain the diff to just these three lines. In order to do that properly, we may need to modify the way height hints are interpreted to mark some as "incomplete". Assuming this goes through, then if we check point these heights, these are now starting points in the backwards search, rather then starting points in the forwards scan. |
In this commit, we modify all existing historical rescans for ChainNotifier backends to scan backwards rather than forwards. If we know that a transaction has been confirmed, or outpoint spent, the it's likely that the event has recently transpired assuming we've been offline for a short period of time. Therefore, if we scan backwards rather than forwards, then we can save potentially hundreds or thousands of block fetches if the event recently happened close to the tip of the chain. We bound this search at the genesis block, to ensure we don't underflow the uint32 used throughout the package in the main loop.
Roasbeef
force-pushed
the
notifier-reverse-scan
branch
from
31ba7bd to
28eb847
Compare
|
Pushed out a new version to fix the test flake. Other option was to modify the tests, but this route allows the loop to not error out w/o a status code on simnet genesis chains. |
|
Updating the height hint caches w/ partial rescan results is tricky, and was left out of the original PRs due to increased complexity. Scanning backwards indeed makes this more complicated if we don't interpret height hint caches as forward-scanning lower bounds. My guess is that with these changes, partial checkpoints won't be necessary as rescan duration should be shortened considerably, but can revisit if lengthy rescans persist |
Roasbeef
added
enhancement
In this commit, we modify all existing historical rescans for
ChainNotifier backends to scan backwards rather than forwards. If we
know that a transaction has been confirmed, or outpoint spent, the it's
likely that the event has recently transpired assuming we've been
offline for a short period of time. Therefore, if we scan backwards
rather than forwards, then we can save potentially hundreds or thousands
of block fetches if the event recently happened close to the tip of the
chain.