refactor: Implement generics for CheckPoint, LocalChain, and spk_client types

[LagginTimes]

Implements #1757.

Description

This PR is a step towards header checkpointing for bdk_electrum. The goal is to be able to store whole headers in CheckPoint so they do not have to be re-downloaded. Storing headers this way would be a prerequisite for caching of merkle proofs and for median time passed.

TODO:

• Change CheckPoint to take in a generic.
• Change LocalChange to take in a generic.
• Change spk_client types to take in generics.

Notes to the reviewers

Changelog notice

• CheckPoint takes in a generic.
• LocalChain and ChangeSet take in generics.
• spk_client types can take in generics.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

[ValuedMammal]

When B is a blockhash, won't this be storing the same block hash twice? 🤔

[nymius]

+1 on this. Maybe data: Option<B> or just keep the height as part of CPInner?

struct CPInner<B> {
  height: u32,
  data: B,
  prev: Option<Arc<CPinner<B>>>,
}

[evanlinjin]

Let's look at this from the API/performance perspective.

I think the "killer feature" of this PR is to be able to put Headers in Checkpoints.

Ideally, we want something like this: CheckPoint<Header>. However, with the proposed changes, it will mean we will be hashing the Header every time we want the block hash from the checkpoint (which will be extremely frequent).

To fix this, we can introduce another data type HeaderWithHash - which, as the name suggests, contains the Header and the BlockHash together. I'm honestly not a fan of this as it becomes clunky.

Since memory is cheap, I think it is okay to duplicate the BlockHash.

[evanlinjin]

If we really don't want to duplicate the BlockHash, another solution would be to use an enum or option internally. It's honestly not pretty and we need to worry about unwraps and how to store the data. Not a fan.

[ValuedMammal]

HeaderWithHash sounds reasonable. (or HashedHeader)

[evanlinjin]

Let's look at this from the API/performance perspective.

I think the "killer feature" of this PR is to be able to put Headers in Checkpoints.

Ideally, we want something like this: CheckPoint<Header>. However, with the proposed changes, it will mean we will be hashing the Header every time we want the block hash from the checkpoint (which will be extremely frequent).

To fix this, we can introduce another data type HeaderWithHash - which, as the name suggests, contains the Header and the BlockHash together. I'm honestly not a fan of this as it becomes clunky.

Since memory is cheap, I think it is okay to duplicate the BlockHash.

[notmandatory]

Since this one is still having design discussions can I move it out of the 1.1 milestone?

[notmandatory]

Took this out of the bdk_wallet 1.1 milestone since tagging the release tomorrow.

[rustaceanrob]

Dropping in to note that this would be incredible useful for bdk_kyoto. It is likely that a LocalChain generic over Header would allow the CBF system to simply use the LocalChain the chain backend, and the persistence may be shared between a Wallet and the node.

[luisschwab]

Dropping in to note that this would be incredible useful for bdk_kyoto. It is likely that a LocalChain generic over Header would allow the CBF system to simply use the LocalChain the chain backend, and the persistence may be shared between a Wallet and the node.

+1 for this feature, bdk_floresta could share persistence with bdk_wallet as well.

[ValuedMammal]

At a high level I think the API would be cleaner if the generic methods had the simpler names (new, push, extend, insert) and the old methods were renamed push_block_id, extend_block_ids, etc.

CheckPoint::data method can be two methods:

fn data_ref(&self) -> &B {}

fn data(&self) -> B where B: Clone {} // or Copy

LocalChain should have more of a unified interface. Most of the methods should be generic and the only methods under impl LocalChain<BlockHash> {} should be the ones that only make sense in the old design (from_genesis_hash, insert_block, etc). Having two versions of apply_changeset and _check_changeset_is_applied isn't very helpful.

I think the duplicate blockhash problem can be fixed using nymius' suggestion to change CheckPoint::block_id member to height.

Remaining questions:

1. Will it be possible to migrate an existing sqlite database to one that persists a local chain of headers, or will it require adding a new SQL table?
2. Are there plans to enable syncing a local chain of headers using bdk_electrum / esplora?