Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable generic type encoding via TypeResolver and remove dependency on scale-info #19

Merged
merged 8 commits into from
Feb 16, 2024
Merged

Enable generic type encoding via TypeResolver and remove dependency on scale-info #19

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
19b53b1
WIP use TypeResolver
jsdw Feb 13, 2024
6c4b4ee
Finish updating to use TypeResolver (minus dep patch)
jsdw Feb 14, 2024
d1dfcf1
update to crates.io scale-type-resolver
jsdw Feb 15, 2024
2a8369d
fix no-std and clippy
jsdw Feb 15, 2024
a727477
remove scale-info references and ensure READMEs remain in sync
jsdw Feb 15, 2024
cdf624c
remove []s in READMEs
jsdw Feb 16, 2024
c4854a7
remove scale-info from dependency tree entirely
jsdw Feb 16, 2024
4126071
Uncomment some tests, tweak a comment, expect over unwrap
jsdw Feb 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,16 @@ jobs:
# we run tests using BitVec<u64,_> which doesn't.
args: --all-features --target wasm32-unknown-unknown

diff-readme-files:
name: Check that READMEs are identical
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3

- name: Diff READMEs
run: diff -q README.md scale-encode/README.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dq: why do we need two readme's?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one in /scale-encode is the oen that is shown on crates.io for the crate, and the one at root is shown on github! This was the laziest way I could think of to make sure that they were kept in sync :)

Copy link
Member

@niklasad1 niklasad1 Feb 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could just link to root/github README in the Cargo.toml of the crate but fine :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, probably a symlink would work here to avoid some extra CI minutes :D

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might not work because if the readme is outside of the crate it won't be uploaded with it, but I'd have to try it!

Copy link
Collaborator Author

@jsdw jsdw Feb 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Git does handle symlinks ok (as long as the FS supports it) though by the sounds of it (eg https://stackoverflow.com/questions/954560/how-does-git-handle-symbolic-links) (I'd assumed that it wouldn't!), so we could just make one readme be a symlink to the other, but I'm not too bothered either way given the CI check :)


no_std:
name: Check no_std build
runs-on: ubuntu-latest
Expand Down
22 changes: 11 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# scale-encode

`parity-scale-codec` provides an `Encode` trait which allows types to SCALE encode themselves based on their shape.
This crate builds on this, and allows types to encode themselves based on `scale_info` type information. It
exposes two traits:
This crate builds on this, and allows types to encode themselves based on type information from a [`TypeResolver`]
implementation (one such implementation being a `scale_info::PortableRegistry`). It exposes two traits:

- An `EncodeAsType` trait which when implemented on some type, describes how it can be SCALE encoded
- An [`EncodeAsType`] trait which when implemented on some type, describes how it can be SCALE encoded
with the help of a type ID and type registry describing the expected shape of the encoded bytes.
- An `EncodeAsFields` trait which when implemented on some type, describes how it can be SCALE encoded
with the help of a slice of `PortableField`'s or `PortableFieldId`'s and type registry describing the
expected shape of the encoded bytes. This is generally only implemented for tuples and structs, since we
need a set of fields to map to the provided slices.
- An [`EncodeAsFields`] trait which when implemented on some type, describes how it can be SCALE encoded
with the help of an iterator over [`Field`]s and a type registry describing the expected shape of the
encoded bytes. This is generally only implemented for tuples and structs, since we need a set of fields
to map to the provided iterator.

Implementations for many built-in types are also provided for each trait, and the `macro@EncodeAsType`
Implementations for many built-in types are also provided for each trait, and the [`macro@EncodeAsType`]
macro makes it easy to generate implementations for new structs and enums.

# Motivation
Expand All @@ -24,14 +24,14 @@ use codec::Encode;
use scale_encode::EncodeAsType;
use scale_info::{PortableRegistry, TypeInfo};

// We are comonly provided type information, but for our examples we construct type info from
// We are commonly provided type information, but for our examples we construct type info from
// any type that implements `TypeInfo`.
fn get_type_info<T: TypeInfo + 'static>() -> (u32, PortableRegistry) {
let m = scale_info::MetaType::new::<T>();
let mut types = scale_info::Registry::new();
let ty = types.register_type(&m);
let portable_registry: PortableRegistry = types.into();
(ty.id(), portable_registry)
(ty.id, portable_registry)
}

// Encode the left value via EncodeAsType into the shape of the right value.
Expand All @@ -43,7 +43,7 @@ where
B: TypeInfo + Encode + 'static,
{
let (type_id, types) = get_type_info::<B>();
let a_bytes = a.encode_as_type(type_id, &types).unwrap();
let a_bytes = a.encode_as_type(&type_id, &types).unwrap();
let b_bytes = b.encode();
assert_eq!(a_bytes, b_bytes);
}
Expand Down
4 changes: 2 additions & 2 deletions scale-encode/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ include.workspace = true
default = ["std", "derive", "primitive-types", "bits"]

# Activates std feature.
std = ["scale-info/std"]
std = []

# Include the derive proc macro.
derive = ["dep:scale-encode-derive"]
Expand All @@ -39,7 +39,7 @@ derive_more = { version = "0.99.17", default-features = false, features = ["from

[dev-dependencies]
bitvec = { version = "1.0.1", default-features = false }
scale-info = { version = "2.3.0", features = ["bit-vec", "derive"], default-features = false }
scale-info = { version = "2.3.0", features = ["bit-vec", "derive", "std"], default-features = false }
scale-encode-derive = { workspace = true }
codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false, features = ["derive", "bit-vec"] }
trybuild = "1.0.72"
Expand Down
22 changes: 11 additions & 11 deletions scale-encode/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# scale-encode

`parity-scale-codec` provides an `Encode` trait which allows types to SCALE encode themselves based on their shape.
This crate builds on this, and allows types to encode themselves based on `scale_info` type information. It
exposes two traits:
This crate builds on this, and allows types to encode themselves based on type information from a [`TypeResolver`]
implementation (one such implementation being a `scale_info::PortableRegistry`). It exposes two traits:

- An `EncodeAsType` trait which when implemented on some type, describes how it can be SCALE encoded
- An [`EncodeAsType`] trait which when implemented on some type, describes how it can be SCALE encoded
with the help of a type ID and type registry describing the expected shape of the encoded bytes.
- An `EncodeAsFields` trait which when implemented on some type, describes how it can be SCALE encoded
with the help of a slice of `PortableField`'s or `PortableFieldId`'s and type registry describing the
expected shape of the encoded bytes. This is generally only implemented for tuples and structs, since we
need a set of fields to map to the provided slices.
- An [`EncodeAsFields`] trait which when implemented on some type, describes how it can be SCALE encoded
with the help of an iterator over [`Field`]s and a type registry describing the expected shape of the
encoded bytes. This is generally only implemented for tuples and structs, since we need a set of fields
to map to the provided iterator.

Implementations for many built-in types are also provided for each trait, and the `macro@EncodeAsType`
Implementations for many built-in types are also provided for each trait, and the [`macro@EncodeAsType`]
macro makes it easy to generate implementations for new structs and enums.

# Motivation
Expand All @@ -24,14 +24,14 @@ use codec::Encode;
use scale_encode::EncodeAsType;
use scale_info::{PortableRegistry, TypeInfo};

// We are comonly provided type information, but for our examples we construct type info from
// We are commonly provided type information, but for our examples we construct type info from
// any type that implements `TypeInfo`.
fn get_type_info<T: TypeInfo + 'static>() -> (u32, PortableRegistry) {
let m = scale_info::MetaType::new::<T>();
let mut types = scale_info::Registry::new();
let ty = types.register_type(&m);
let portable_registry: PortableRegistry = types.into();
(ty.id(), portable_registry)
(ty.id, portable_registry)
}

// Encode the left value via EncodeAsType into the shape of the right value.
Expand All @@ -43,7 +43,7 @@ where
B: TypeInfo + Encode + 'static,
{
let (type_id, types) = get_type_info::<B>();
let a_bytes = a.encode_as_type(type_id, &types).unwrap();
let a_bytes = a.encode_as_type(&type_id, &types).unwrap();
let b_bytes = b.encode();
assert_eq!(a_bytes, b_bytes);
}
Expand Down
6 changes: 3 additions & 3 deletions scale-encode/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@

/*!
`parity-scale-codec` provides an `Encode` trait which allows types to SCALE encode themselves based on their shape.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dq: Could we #[include_str(README.md)] here?

Copy link
Collaborator Author

@jsdw jsdw Feb 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could; I think the only reason I don't do that is because the lib docs benefit from doc links ([Foo]) whereas in the README it's better to remove the []'s!

There's almost certainly a tool somewhere though which takes the lib docs and copies them into a README for you, stripping such links!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense, the only crate I can think of might be: https://crates.io/crates/docify, something that substrate's been using for pallet docs

This crate builds on this, and allows types to encode themselves based on [`scale_info`] type information. It
exposes two traits:
This crate builds on this, and allows types to encode themselves based on type information from a [`TypeResolver`]
implementation (one such implementation being a `scale_info::PortableRegistry`). It exposes two traits:

- An [`EncodeAsType`] trait which when implemented on some type, describes how it can be SCALE encoded
with the help of a type ID and type registry describing the expected shape of the encoded bytes.
Expand Down Expand Up @@ -164,7 +164,7 @@ pub mod ext {
}

/// This trait signals that some static type can possibly be SCALE encoded given some
/// `type_id` and [`PortableRegistry`] which dictates the expected encoding.
/// `type_id` and a corresponding [`TypeResolver`] which tells us about the expected encoding.
pub trait EncodeAsType {
/// Given some `type_id`, `types`, a `context` and some output target for the SCALE encoded bytes,
/// attempt to SCALE encode the current value into the type given by `type_id`.
Expand Down
Loading