Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc: event subscription management for RPC client #516

Merged
merged 61 commits into from
Sep 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
9df13ed
Create basic request/response transport abstraction
thanethomson Aug 10, 2020
a01883d
Add fixture-based transport with failing test
thanethomson Aug 10, 2020
a03d0ab
Refactor mock transport
thanethomson Aug 11, 2020
73c2626
Take client creation interface change into account
thanethomson Aug 11, 2020
8650db1
Reword docstring for Transport
thanethomson Aug 11, 2020
cf67215
Add initial subscription management mechanism
thanethomson Aug 12, 2020
20cce0a
Ignore integration test
thanethomson Aug 12, 2020
92cae5f
Update rpc/Cargo.toml
thanethomson Aug 12, 2020
40e7de2
Update rpc/src/client.rs
thanethomson Aug 12, 2020
4634e48
Update rpc/src/client/subscription.rs
thanethomson Aug 12, 2020
82cf571
Update rpc/src/client/subscription.rs
thanethomson Aug 12, 2020
52c36d0
Update rpc/src/client/subscription.rs
thanethomson Aug 12, 2020
1089e8d
Update rpc/src/client/transport.rs
thanethomson Aug 12, 2020
5994999
Simplify submodule naming
thanethomson Aug 12, 2020
1ef2305
Reorder optional dependencies alphabetically
thanethomson Aug 12, 2020
a364187
Rename module to singular
thanethomson Aug 12, 2020
15d5765
Destructure and reformat imports
thanethomson Aug 12, 2020
7e6d15d
Refactor subscription mechanism to simplify
thanethomson Aug 13, 2020
da0247a
Add note for myself on TODO
thanethomson Aug 13, 2020
cafa8d7
Move mod declarations before use statements
thanethomson Aug 13, 2020
9ad109b
Fix clippy warnings
thanethomson Aug 13, 2020
ecb099f
Refactor interface entirely
thanethomson Aug 14, 2020
cf56f7b
Reduce number of blocks grabbed to speed up test
thanethomson Aug 14, 2020
ae66f90
Improve documentation for RPC client
thanethomson Aug 14, 2020
d3d2c32
Fix links in method docs
thanethomson Aug 14, 2020
f01d13d
Expose client docs in base RPC package
thanethomson Aug 14, 2020
279913b
Refactor to allow for remote errors
thanethomson Aug 16, 2020
6d35180
Correctly produce a subscription error for possible RPC protocol mism…
thanethomson Aug 16, 2020
2228b71
Remove unnecessary field
thanethomson Aug 16, 2020
8bc9f7f
Fix unsubscribe mechanism
thanethomson Aug 16, 2020
f017d27
Fix failing doctest
thanethomson Aug 16, 2020
233fd62
Allow for PartialEq comparisons (to aid in testing)
thanethomson Aug 16, 2020
36e2969
Add tests for SubscriptionRouter
thanethomson Aug 16, 2020
83d4835
Remove code no longer used
thanethomson Aug 16, 2020
ab4ba9b
result module need not be public
thanethomson Aug 18, 2020
b160e65
Clean up docs
thanethomson Aug 18, 2020
86ed3d9
Rename `ClientError` to `ClientInternalError` and update docs
thanethomson Aug 18, 2020
fd01278
Fix typo
thanethomson Aug 18, 2020
3eadccd
Not using nightly docsrs features yet
thanethomson Aug 18, 2020
a4c5a4d
Refactor and restructure interface
thanethomson Aug 26, 2020
1c82638
Minor documentation fixes
thanethomson Aug 26, 2020
772bcd0
Add newline at end of JSON fixtures
thanethomson Aug 26, 2020
1d08a7d
Remove request::Wrapper::new_with_id() method
thanethomson Aug 26, 2020
3048155
Refactor interface for subscription client
thanethomson Aug 27, 2020
5e9e351
Fix broken link in documentation
thanethomson Aug 27, 2020
99dae2a
Rename all "JSONRPC" references to "JSON-RPC"
thanethomson Aug 27, 2020
ba2ca31
Reword docs for SubscriptionClient
thanethomson Aug 27, 2020
b4253c6
TODO is no longer necessary since we are using unbounded channels
thanethomson Aug 27, 2020
b9e240e
Update CHANGELOG
thanethomson Aug 27, 2020
99a321f
Clarify docs for Subscription::terminate
thanethomson Aug 27, 2020
3e38b0b
Merge branch 'master' into rpc/subscription
thanethomson Sep 14, 2020
58d52ef
Fix features for tendermint-rpc dependency in light client
thanethomson Sep 15, 2020
e218dd6
Fix clippy warnings
thanethomson Sep 15, 2020
ee70331
Do not feature-guard IoError::IoError variant
romac Sep 15, 2020
e7bb6a1
Rename IoError::IoError to IoError::RpcError
romac Sep 15, 2020
a891c21
Refactor subscription state modelling
thanethomson Sep 15, 2020
c22a5da
Replace SubscriptionId AsRef implementation with as_str() method for …
thanethomson Sep 15, 2020
1753566
Rename TxResult and TxResultResult for clarity
thanethomson Sep 15, 2020
4db8d9e
Add explanation of SubscriptionRouter subscriptions field
thanethomson Sep 15, 2020
e293693
Comment on assumption regarding handling of event publishing failure
thanethomson Sep 15, 2020
e7403a6
Add comment explaining redeclaration and obtaining of local var
thanethomson Sep 15, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,41 @@
- Separate protobuf types from Rust domain types using the DomainType trait ([#535])
- Changed validator sorting order to sort by voting power. ([#506])

### BREAKING CHANGES:

- `[rpc]` The entire RPC client interface has been refactored. The
`Client` struct has now been replaced by an `HttpClient` struct, which
implements all of the RPC methods except those relating to event
subscription. To access this struct, you now need to enable both the
`client` and `transport_http` features when using the `tendermint-rpc`
crate. ([#516])

### IMPROVEMENTS:

- `[rpc]` A `WebSocketSubscriptionClient` is now provided to facilitate event
subscription for a limited range of RPC events over a WebSocket connection.
See the [Tendermint `/subscribe` endpoint's](https://docs.tendermint.com/master/rpc/#/Websocket/subscribe)
and the `tendermint-rpc` crate's docs for more details.
To access this struct you need to enable both the `client`, `subscription`
and `transport_websocket` features when using the `tendermint-rpc` crate.
([#516])
- `[rpc]` A `MockClient` and `MockSubscriptionClient` struct are available for use in
instances where you may want to interact with the Tendermint RPC from your
tests without integrating with an actual node. To access these structs you
need to enable the `client`, `subscription` and `transport_mock` features
when using the `tendermint-rpc` crate. If you only want to use the
`MockClient` struct, just enable features `client` and `transport_mock`.
See the crate docs for more details.
([#516])

[#524]: https://github.com/informalsystems/tendermint-rs/issues/524
[#526]: https://github.com/informalsystems/tendermint-rs/issues/526
[#498]: https://github.com/informalsystems/tendermint-rs/issues/498
[#463]: https://github.com/informalsystems/tendermint-rs/issues/463
[#504]: https://github.com/informalsystems/tendermint-rs/issues/504
[#535]: https://github.com/informalsystems/tendermint-rs/issues/535
[#506]: https://github.com/informalsystems/tendermint-rs/issues/506
[#516]: https://github.com/informalsystems/tendermint-rs/pull/516

## v0.16.0

Expand Down
2 changes: 1 addition & 1 deletion light-client/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ crate-type = ["cdylib", "rlib"]

[features]
default = ["rpc-client"]
rpc-client = ["tendermint-rpc/client"]
rpc-client = ["tendermint-rpc/client", "tendermint-rpc/transport_http"]
secp256k1 = ["tendermint/secp256k1", "tendermint-rpc/secp256k1"]

[dependencies]
Expand Down
18 changes: 11 additions & 7 deletions light-client/src/components/io.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ use contracts::{contract_trait, post};
use serde::{Deserialize, Serialize};
use thiserror::Error;

#[cfg(feature = "rpc-client")]
use tendermint_rpc::Client;

use tendermint_rpc as rpc;

use crate::types::{Height, LightBlock, PeerId};
Expand Down Expand Up @@ -31,7 +34,7 @@ impl From<Height> for AtHeight {
pub enum IoError {
/// Wrapper for a `tendermint::rpc::Error`.
#[error(transparent)]
IoError(#[from] rpc::Error),
RpcError(#[from] rpc::Error),

/// Given height is invalid
#[error("invalid height: {0}")]
Expand Down Expand Up @@ -128,7 +131,7 @@ mod prod {
peer: PeerId,
height: AtHeight,
) -> Result<TMSignedHeader, IoError> {
let rpc_client = self.rpc_client_for(peer);
let rpc_client = self.rpc_client_for(peer)?;

let res = block_on(
async {
Expand All @@ -143,7 +146,7 @@ mod prod {

match res {
Ok(response) => Ok(response.signed_header),
Err(err) => Err(IoError::IoError(err)),
Err(err) => Err(IoError::RpcError(err)),
}
}

Expand All @@ -161,21 +164,22 @@ mod prod {
};

let res = block_on(
self.rpc_client_for(peer).validators(height),
self.rpc_client_for(peer)?.validators(height),
peer,
self.timeout,
)?;

match res {
Ok(response) => Ok(TMValidatorSet::new(response.validators)),
Err(err) => Err(IoError::IoError(err)),
Err(err) => Err(IoError::RpcError(err)),
}
}

// TODO(thane): Generalize over client transport (instead of using HttpClient directly).
ebuchman marked this conversation as resolved.
Show resolved Hide resolved
#[pre(self.peer_map.contains_key(&peer))]
fn rpc_client_for(&self, peer: PeerId) -> rpc::Client {
fn rpc_client_for(&self, peer: PeerId) -> Result<rpc::HttpClient, IoError> {
let peer_addr = self.peer_map.get(&peer).unwrap().to_owned();
rpc::Client::new(peer_addr)
Ok(rpc::HttpClient::new(peer_addr).map_err(IoError::from)?)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe pedantic but shouldn't the PeerId type already hold a valid address and hence this shouldn't need to return an error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically a Tendermint Address could contain either a TCP or a Unix socket address, and we need the host/port details in order to connect to the RPC endpoint.

We could do away with the need for an error here by requiring host/port details instead of a peer_addr, or we could just panic if a user supplies a Unix socket address?

}
}

Expand Down
9 changes: 5 additions & 4 deletions light-client/src/evidence.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ mod prod {
use contracts::pre;
use std::collections::HashMap;
use tendermint_rpc as rpc;
use tendermint_rpc::Client;

/// Production implementation of the EvidenceReporter component, which reports evidence to full
/// nodes via RPC.
Expand All @@ -38,11 +39,11 @@ mod prod {
impl EvidenceReporter for ProdEvidenceReporter {
#[pre(self.peer_map.contains_key(&peer))]
fn report(&self, e: Evidence, peer: PeerId) -> Result<Hash, IoError> {
let res = block_on(self.rpc_client_for(peer).broadcast_evidence(e));
let res = block_on(self.rpc_client_for(peer)?.broadcast_evidence(e));

match res {
Ok(response) => Ok(response.hash),
Err(err) => Err(IoError::IoError(err)),
Err(err) => Err(IoError::RpcError(err)),
}
}
}
Expand All @@ -56,9 +57,9 @@ mod prod {
}

#[pre(self.peer_map.contains_key(&peer))]
fn rpc_client_for(&self, peer: PeerId) -> rpc::Client {
fn rpc_client_for(&self, peer: PeerId) -> Result<rpc::HttpClient, IoError> {
let peer_addr = self.peer_map.get(&peer).unwrap().to_owned();
rpc::Client::new(peer_addr)
Ok(rpc::HttpClient::new(peer_addr).map_err(IoError::from)?)
}
}

Expand Down
2 changes: 1 addition & 1 deletion light-node/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ description = """
The Tendermint light-node wraps the light-client crate into a command-line
interface tool.
It can be used to initialize and start a standalone light client daemon and
exposes a JSONRPC endpoint from which you can query the current state of the
exposes a JSON-RPC endpoint from which you can query the current state of the
light node.
"""

Expand Down
4 changes: 2 additions & 2 deletions light-node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ See the [repo root] for build status, license, rust version, etc.
# Light-Node

The [Tendermint] light-node wraps the [light-client] crate into a command-line interface tool.
It can be used as a standalone light client daemon and exposes a JSONRPC endpoint
It can be used as a standalone light client daemon and exposes a JSON-RPC endpoint
from which you can query the current state of the light node.

## Getting Started
Expand Down Expand Up @@ -103,7 +103,7 @@ Or on a specific sub-command, e.g.:
$ cargo run -- help start
```

### JSONRPC Endpoint(s)
### JSON-RPC Endpoint(s)

When you have a light-node running you can query its current state via:
```
Expand Down
2 changes: 1 addition & 1 deletion light-node/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
//!
//! The Tendermint light-node wraps the light-client crate into a command-line interface tool.
//!
//! It can be used to initialize and start a standalone light client daemon and exposes a JSONRPC
//! It can be used to initialize and start a standalone light client daemon and exposes a JSON-RPC
//! endpoint from which you can query the current state of the light node.

// Tip: Deny warnings with `RUSTFLAGS="-D warnings"` environment variable in CI
Expand Down
2 changes: 1 addition & 1 deletion light-node/src/rpc.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//! JSONRPC Server and Client for the light-node RPC endpoint.
//! JSON-RPC Server and Client for the light-node RPC endpoint.
use jsonrpc_core::IoHandler;
use jsonrpc_http_server::{AccessControlAllowOrigin, DomainsValidation, ServerBuilder};

Expand Down
17 changes: 11 additions & 6 deletions rpc/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ authors = [
]

description = """
tenndermint-rpc contains the core types returned by a Tendermint node's RPC endpoint.
tendermint-rpc contains the core types returned by a Tendermint node's RPC endpoint.
All networking related features are feature guarded to keep the dependencies small in
cases where only the core types are needed.
"""
Expand All @@ -24,9 +24,13 @@ description = """
all-features = true

[features]
default = ["client"]
client = ["async-tungstenite", "futures", "http", "hyper", "tokio"]
secp256k1 = ["tendermint/secp256k1"]
default = []
client = [ "async-trait", "futures" ]
secp256k1 = [ "tendermint/secp256k1" ]
subscription = [ "tokio" ]
transport_http = [ "http", "hyper", "tokio" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should just be consolidated with client? How much benefit do we really get by opting for the transport_mock over the transport_http ?

transport_mock = [ "tokio" ]
transport_websocket = [ "async-tungstenite", "tokio" ]

[dependencies]
bytes = "0.5"
Expand All @@ -38,8 +42,9 @@ tendermint = { version = "0.16.0", path = "../tendermint" }
thiserror = "1"
uuid = { version = "0.8", default-features = false }

async-tungstenite = { version="0.5", features = ["tokio-runtime"], optional = true }
async-tungstenite = { version="0.8", features = ["tokio-runtime"], optional = true }
async-trait = { version = "0.1", optional = true }
futures = { version = "0.3", optional = true }
http = { version = "0.2", optional = true }
hyper = { version = "0.13", optional = true }
tokio = { version = "0.2", features = ["macros"], optional = true }
tokio = { version = "0.2", features = ["macros", "fs", "sync"], optional = true }
2 changes: 1 addition & 1 deletion rpc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ See the [repo root] for build status, license, rust version, etc.

A rust implementation of the core types returned by a Tendermint node's RPC
endpoint.
These can be used to deserialize JSONRPC responses.
These can be used to deserialize JSON-RPC responses.
All networking related features will be feature guarded to keep the dependencies small
in cases where only the core types are needed.

Expand Down
Loading