Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RPC-Spec-V2] chainHead: use integer for block index and adjust RuntimeVersion JSON format #1666

Merged
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions substrate/client/rpc-spec-v2/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ tokio = { version = "1.22.0", features = ["sync"] }
array-bytes = "6.1"
log = "0.4.17"
futures-util = { version = "0.3.19", default-features = false }
impl-serde = { version = "0.4.0", default-features = false }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it still be necessary to have this extra dependency if we use a serde(from = ... and serde(into = for the RuntimeVersion? 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we would still need it, because we would still need to define how ApiId (which is a [u8; 8]) is serialized (as a string). The serialization of RuntimeVersion also relies on a custom serialization using impl-serde, so we would ahve to do something similar for our wrapper.


[dev-dependencies]
serde_json = "1.0"
Expand Down
1 change: 0 additions & 1 deletion substrate/client/rpc-spec-v2/src/chain_head/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@ impl From<Error> for ErrorObject<'static> {
Error::InvalidSubscriptionID => ErrorObject::owned(INVALID_SUB_ID, msg, None::<()>),
Error::InvalidContinue => ErrorObject::owned(INVALID_CONTINUE, msg, None::<()>),
}
.into()
}
}

Expand Down
155 changes: 151 additions & 4 deletions substrate/client/rpc-spec-v2/src/chain_head/event.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ pub struct ErrorEvent {
#[serde(rename_all = "camelCase")]
pub struct RuntimeVersionEvent {
/// The runtime version.
#[serde(
serialize_with = "custom_serialize::serialize_runtime_version",
deserialize_with = "custom_serialize::deserialize_runtime_version"
)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do wonder if we could take another approach here, since implementing deserialization is quite complicated in general:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lexnv Good idea, would definitely be simpler! I don't know if it is important, but this would come at a slightly higher runtime cost because we would construct a HashMap every time we create a RuntimeVersionWrapper from a RuntimeVersion before serializing it.

pub spec: RuntimeVersion,
}

Expand Down Expand Up @@ -349,6 +353,148 @@ pub struct MethodResponseStarted {
pub discarded_items: Option<usize>,
}

mod custom_serialize {
use serde::{de, ser::SerializeMap, Deserialize, Serialize, Serializer};

pub fn serialize_runtime_version<S>(
runtime_version: &sp_version::RuntimeVersion,
ser: S,
) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let representation = RuntimeVersionV2::from(runtime_version);
representation.serialize(ser)
}

pub fn deserialize_runtime_version<'de, D>(
deserializer: D,
) -> Result<sp_version::RuntimeVersion, D::Error>
where
D: de::Deserializer<'de>,
{
let representation = RuntimeVersionV2Owned::deserialize(deserializer)?;
Ok(representation.into())
}

#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
struct RuntimeVersionV2<'a> {
spec_name: &'a str,
impl_name: &'a str,
spec_version: u32,
impl_version: u32,
#[serde(serialize_with = "serialize_apis_vec_as_map")]
apis: &'a sp_version::ApisVec,
transaction_version: u32,
}

impl<'a> From<&'a sp_version::RuntimeVersion> for RuntimeVersionV2<'a> {
fn from(value: &'a sp_version::RuntimeVersion) -> Self {
RuntimeVersionV2 {
spec_name: &value.spec_name,
impl_name: &value.impl_name,
spec_version: value.spec_version,
impl_version: value.impl_version,
apis: &value.apis,
transaction_version: value.transaction_version,
}
}
}

#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct RuntimeVersionV2Owned {
spec_name: sp_runtime::RuntimeString,
impl_name: sp_runtime::RuntimeString,
spec_version: u32,
impl_version: u32,
#[serde(deserialize_with = "deserialize_apis_vec_as_map")]
apis: sp_version::ApisVec,
transaction_version: u32,
}

impl From<RuntimeVersionV2Owned> for sp_version::RuntimeVersion {
fn from(value: RuntimeVersionV2Owned) -> Self {
sp_version::RuntimeVersion {
spec_name: value.spec_name,
impl_name: value.impl_name,
spec_version: value.spec_version,
impl_version: value.impl_version,
apis: value.apis,
transaction_version: value.transaction_version,
..Default::default()
}
}
}

pub fn serialize_apis_vec_as_map<S>(apis: &[([u8; 8], u32)], ser: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let len = apis.len();
let mut map = ser.serialize_map(Some(len))?;
for (api, ver) in apis.iter() {
let api = ApiId(api);
map.serialize_entry(&api, ver)?;
}
map.end()
}

pub fn deserialize_apis_vec_as_map<'de, D>(
deserializer: D,
) -> Result<sp_version::ApisVec, D::Error>
where
D: de::Deserializer<'de>,
{
struct Visitor;
impl<'de> de::Visitor<'de> for Visitor {
type Value = sp_version::ApisVec;

fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
formatter.write_str("a map from api id to version")
}

fn visit_map<V>(self, mut visitor: V) -> Result<Self::Value, V::Error>
where
V: de::MapAccess<'de>,
{
let mut apis = Vec::new();
while let Some((key, value)) = visitor.next_entry::<ApiIdOwned, u32>()? {
apis.push((key.0, value));
}
Ok(apis.into())
}
}
deserializer.deserialize_map(Visitor)
}

#[derive(Serialize)]
struct ApiId<'a>(#[serde(serialize_with = "serialize_api_id")] &'a sp_version::ApiId);

#[derive(Deserialize)]
struct ApiIdOwned(#[serde(deserialize_with = "deserialize_api_id")] sp_version::ApiId);

pub fn serialize_api_id<S>(&api_id: &&sp_version::ApiId, ser: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
impl_serde::serialize::serialize(api_id, ser)
}

pub fn deserialize_api_id<'de, D>(d: D) -> Result<sp_version::ApiId, D::Error>
where
D: de::Deserializer<'de>,
{
let mut arr = [0; 8];
impl_serde::serialize::deserialize_check_len(
d,
impl_serde::serialize::ExpectedLen::Exact(&mut arr[..]),
)?;
Ok(arr)
}
}

#[cfg(test)]
mod tests {
use super::*;
Expand Down Expand Up @@ -391,8 +537,8 @@ mod tests {
let ser = serde_json::to_string(&event).unwrap();
let exp = concat!(
r#"{"event":"initialized","finalizedBlockHash":"0x1","#,
r#""finalizedBlockRuntime":{"type":"valid","spec":{"specName":"ABC","implName":"Impl","authoringVersion":0,"#,
r#""specVersion":1,"implVersion":0,"apis":[],"transactionVersion":0,"stateVersion":0}}}"#,
r#""finalizedBlockRuntime":{"type":"valid","spec":{"specName":"ABC","implName":"Impl","#,
r#""specVersion":1,"implVersion":0,"apis":{},"transactionVersion":0}}}"#,
);
assert_eq!(ser, exp);

Expand Down Expand Up @@ -429,6 +575,7 @@ mod tests {
spec_name: "ABC".into(),
impl_name: "Impl".into(),
spec_version: 1,
apis: vec![([0, 0, 0, 0, 0, 0, 0, 0], 2), ([1, 0, 0, 0, 0, 0, 0, 0], 3)].into(),
..Default::default()
};

Expand All @@ -445,8 +592,8 @@ mod tests {
let ser = serde_json::to_string(&event).unwrap();
let exp = concat!(
r#"{"event":"newBlock","blockHash":"0x1","parentBlockHash":"0x2","#,
r#""newRuntime":{"type":"valid","spec":{"specName":"ABC","implName":"Impl","authoringVersion":0,"#,
r#""specVersion":1,"implVersion":0,"apis":[],"transactionVersion":0,"stateVersion":0}}}"#,
r#""newRuntime":{"type":"valid","spec":{"specName":"ABC","implName":"Impl","#,
r#""specVersion":1,"implVersion":0,"apis":{"0x0000000000000000":2,"0x0100000000000000":3},"transactionVersion":0}}}"#,
);
assert_eq!(ser, exp);

Expand Down
4 changes: 2 additions & 2 deletions substrate/client/rpc-spec-v2/src/chain_head/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -234,12 +234,12 @@ async fn follow_with_runtime() {
let event: FollowEvent<String> = get_next_event(&mut sub).await;

// it is basically json-encoded substrate_test_runtime_client::runtime::VERSION
let runtime_str = "{\"specName\":\"test\",\"implName\":\"parity-test\",\"authoringVersion\":1,\
let runtime_str = "{\"specName\":\"test\",\"implName\":\"parity-test\",\"authoringVersion\":0,\
\"specVersion\":2,\"implVersion\":2,\"apis\":[[\"0xdf6acb689907609b\",4],\
[\"0x37e397fc7c91f5e4\",2],[\"0xd2bc9897eed08f15\",3],[\"0x40fe3ad401f8959a\",6],\
[\"0xbc9d89904f5b923f\",1],[\"0xc6e9a76309f39b09\",2],[\"0xdd718d5cc53262d4\",1],\
[\"0xcbca25e39f142387\",2],[\"0xf78b278be53f454c\",2],[\"0xab3c0572291feb8b\",1],\
[\"0xed99c5acb25eedf5\",3],[\"0xfbc577b9d747efd6\",1]],\"transactionVersion\":1,\"stateVersion\":1}";
[\"0xed99c5acb25eedf5\",3],[\"0xfbc577b9d747efd6\",1]],\"transactionVersion\":1,\"stateVersion\":0}";

let runtime: RuntimeVersion = serde_json::from_str(runtime_str).unwrap();

Expand Down
5 changes: 2 additions & 3 deletions substrate/client/rpc-spec-v2/src/transaction/event.rs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TransactionBroadcasted would also need to be kept in sync with the number formatting.

We could remove the as_string altogether for now and adjust the testing 👍

Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ pub struct TransactionBlock<Hash> {
/// The hash of the block the transaction was included into.
pub hash: Hash,
/// The index (zero-based) of the transaction within the body of the block.
#[serde(with = "as_string")]
pub index: usize,
}

Expand Down Expand Up @@ -288,7 +287,7 @@ mod tests {
}));
let ser = serde_json::to_string(&event).unwrap();

let exp = r#"{"event":"bestChainBlockIncluded","block":{"hash":"0x0000000000000000000000000000000000000000000000000000000000000001","index":"2"}}"#;
let exp = r#"{"event":"bestChainBlockIncluded","block":{"hash":"0x0000000000000000000000000000000000000000000000000000000000000001","index":2}}"#;
assert_eq!(ser, exp);

let event_dec: TransactionEvent<H256> = serde_json::from_str(exp).unwrap();
Expand All @@ -303,7 +302,7 @@ mod tests {
});
let ser = serde_json::to_string(&event).unwrap();

let exp = r#"{"event":"finalized","block":{"hash":"0x0000000000000000000000000000000000000000000000000000000000000001","index":"10"}}"#;
let exp = r#"{"event":"finalized","block":{"hash":"0x0000000000000000000000000000000000000000000000000000000000000001","index":10}}"#;
assert_eq!(ser, exp);

let event_dec: TransactionEvent<H256> = serde_json::from_str(exp).unwrap();
Expand Down
12 changes: 6 additions & 6 deletions substrate/client/rpc/src/state/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -513,12 +513,12 @@ async fn should_return_runtime_version() {
let (api, _child) = new_full(client.clone(), test_executor(), DenyUnsafe::No);

// it is basically json-encoded substrate_test_runtime_client::runtime::VERSION
let result = "{\"specName\":\"test\",\"implName\":\"parity-test\",\"authoringVersion\":1,\
\"specVersion\":2,\"implVersion\":2,\"apis\":[[\"0xdf6acb689907609b\",4],\
[\"0x37e397fc7c91f5e4\",2],[\"0xd2bc9897eed08f15\",3],[\"0x40fe3ad401f8959a\",6],\
[\"0xbc9d89904f5b923f\",1],[\"0xc6e9a76309f39b09\",2],[\"0xdd718d5cc53262d4\",1],\
[\"0xcbca25e39f142387\",2],[\"0xf78b278be53f454c\",2],[\"0xab3c0572291feb8b\",1],\
[\"0xed99c5acb25eedf5\",3],[\"0xfbc577b9d747efd6\",1]],\"transactionVersion\":1,\"stateVersion\":1}";
let result = "{\"specName\":\"test\",\"implName\":\"parity-test\",
\"specVersion\":2,\"implVersion\":2,\"apis\":{\"0xdf6acb689907609b\":4,
\"0x37e397fc7c91f5e4\":2,\"0xd2bc9897eed08f15\":3,\"0x40fe3ad401f8959a\":6,
\"0xbc9d89904f5b923f\":1,\"0xc6e9a76309f39b09\":2,\"0xdd718d5cc53262d4\":1,
\"0xcbca25e39f142387\":2,\"0xf78b278be53f454c\":2,\"0xab3c0572291feb8b\":1,
\"0xed99c5acb25eedf5\":3,\"0xfbc577b9d747efd6\":1},\"transactionVersion\":1}";

let runtime_version = api.runtime_version(None.into()).unwrap();
let serialized = serde_json::to_string(&runtime_version).unwrap();
Expand Down