Retry logic

[bruwozniak]

Hi, thanks for this project.
Perhaps I've missed it but is there a way to setup any retry logic within the client? Or is this left up to the user to wrap the calls in some retry code?

[niklasad1]

Hey @bruwozniak

You are correct, this project doesn't provide any retry logic so it's up to the user the handle that.

I guess we could add support for tower middleware in the clients to support that as we already have it in the servers.

[seunlanlege]

You are correct, this project doesn't provide any retry logic so it's up to the user the handle that.

Disagree with this retry logic is a bare minimum for any websocket client due to fundamentally unstable nature of websocket connections. The lack of connection retries is making subxt unusable.

[niklasad1]

Ok, for subxt it makes sense to implement some "reconnect strategy" but I'm not convinced to add this logic to jsonrpsee because it's more flexible for users decide that themselves but I will have look it again.

Roughly one have to something as the following to implement a retry strategy on-top of jsonrpsee:

	let mut client = Arc::new(WsClientBuilder::default().build(&url).await?);
	let mut pending_requests = FuturesUnordered::new();

	let c = client.clone();
	pending_requests.push(async move { c.request("say_hello", rpc_params![]).await });

	loop {
		tokio::select! {
			// reconnect on disconnect
			_ = client.on_disconnect() => {
				// All pending calls will most likely fail and one would probably want to try them again after
				// reconnecting. Omitting that for this example
				//
				// TODO: Implement my own reconnect strategy....
				client = Arc::new(WsClientBuilder::default().build(&url).await?);
			}
			_ = pending_requests.next() => {}
		}
	}

[xlc]

This is our client with retry logic https://github.com/AcalaNetwork/subway/blob/master/src/extensions/client/mod.rs

[niklasad1]

There are some things that are not clear how to deal with such as if the active subscriptions should be re-established on disconnect or not. I believe the general case should re-establish subscriptions and re-send the pending calls but it's application dependent.

Might be more cases that I'm missing... that's why we didn't add this in the first place because the API becomes quite messy to configure the retry strategy, what to do with pending calls and active subscriptions.

[seunlanlege]

Might be more cases that I'm missing... that's why we didn't add this in the first place because the API becomes quite messy to configure the retry strategy, what to do with pending calls and active subscriptions.

The retry logic means that the client always guarantees message delivery. You can have different retry policies. Exponential back off, constant or never.

[niklasad1]

Yes, retry logic == retry strategy but message delivery from the client's point-of-view doesn't work well for the subscription-model as I tried to rant about i.e, once these are established these are just "notification messages" sent from the server-side.

To summarize the tricky parts on re-connect:

• Re-establish subscriptions or not
• Subscription notifications may be missed when reconnecting which may crucial for some applications

[seunlanlege]

• Re-establish subscriptions or not

Yes subscriptions should be re-established

• Subscription notifications may be missed when reconnecting which may crucial for some applications

It's better the applications miss some than the connection be terminated outrightly

[niklasad1]

I agree with you in the "general case" but for example in substrate if the connection is lost while subscribing to author_submitAndWatch then it's possible that the Finalized event is missed when reconnecting and a transaction may be submitted twice.

Sure, it's a rare edge-case but possible...

I have opened paritytech/subxt#1190 to continue the discussion around retry strategy in subxt.

[niklasad1]

I have written a library if you want a automatic reconnecting ws client