Fix skipped finalized blocks under EventSubscription

[paulormart]

Hi,

With the latest Runtime/9170 enacted already on Kusama not all the finalized events are emitted, see PR paritytech/substrate#10639 (comment).

I wonder if a workaround could be implemented inside this library rather than on the client side?

Ideally let mut event_sub = api.events().subscribe_finalized().await?; should always guarantee that the last finalized block is the next one in the iterator.

Currently i'm playing with something around these lines to always get the latest events for all the finalized blocks:

use futures::StreamExt;
use subxt::{
    ClientBuilder,
    DefaultConfig,
    DefaultExtra,
};

#[subxt::subxt(runtime_metadata_path = "examples/kusama_metadata.scale")]
pub mod polkadot {}

/// Subscribe to all events, and then manually look through them and
/// pluck out the events that we care about.
#[async_std::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    env_logger::init();

    let api = ClientBuilder::new()
        .set_url("wss://kusama-rpc.polkadot.io:443")
        .build()
        .await?
        .to_runtime_api::<polkadot::RuntimeApi<DefaultConfig, DefaultExtra<DefaultConfig>>>();

    // Subscribe to any events that occur:
    let mut event_sub = api.events().subscribe_finalized().await?;


    let finalized_hash = api.client.rpc().finalized_head().await?;

    let mut last_finalized_block_number = match api.client.rpc().block(Some(finalized_hash)).await? {
        Some(signed_block) => Some(signed_block.block.header.number),
        None => Some(11650722)
    };
    
    // Our subscription will see the events emitted as a result of this:
    while let Some(events) = event_sub.next().await {

        let events = events?;
        let block_hash = events.block_hash();

        if let Some(signed_block) = api.client.rpc().block(Some(block_hash)).await? {
            
            let block_number_finalized = signed_block.block.header.number;

            // Verify if any finalized block has been skipped
            while let Some(i) = last_finalized_block_number {
                let next_block_number = i + 1;
                if next_block_number > block_number_finalized {
                    println!("Wait for the last finalized block");
                    last_finalized_block_number = None;
                } else if next_block_number == block_number_finalized {
                    println!("Do whatever you want with the current events for block {}", block_number_finalized);
                    last_finalized_block_number = Some(block_number_finalized);
                } else {
                    println!("Fetch the events for the skipped block {}", next_block_number);
                    last_finalized_block_number = Some(next_block_number);
                }
            }

            // Set latest block processed and wait for a new finalized block
            last_finalized_block_number = Some(block_number_finalized);

        }        
        
    }

    Ok(())
}

You can see the skipped blocks below:

Wait for the last finalized block
Do whatever you want with the current events for block 11662409
Wait for the last finalized block
Do whatever you want with the current events for block 11662410
Wait for the last finalized block
Fetch the events for the skipped block 11662411
Do whatever you want with the current events for block 11662412
Wait for the last finalized block
Do whatever you want with the current events for block 11662413
Wait for the last finalized block
Do whatever you want with the current events for block 11662414
Wait for the last finalized block
Do whatever you want with the current events for block 11662415
Wait for the last finalized block
Fetch the events for the skipped block 11662416
Do whatever you want with the current events for block 11662417
Wait for the last finalized block

[jsdw]

So if I understand this right, the issue is that when multiple blocks are finalized at once, in the linked PR subscribers are now notified of only the last finalised head (and previously it sounds like they'd be notified of all blocks, up to a limit of 256 blocks).

It also sounds like the finalized details will include a list of all blocks finalised to this point, so internally we could just iterate over each of those I think to obtain the event details from each.

[paulormart]

Hi @jsdw, yes you got it right.

Do you think it makes sense to solve this inside this library?

[jsdw]

I reckon so; if somebody asks to subscribe to finalized events, it seems reasonable to me that they'd expect to be handed back events from all finalized blocks, and not miss some out (partly just to be consistent with how subscribing to non finalized events works).