light-client: Add experimental light-client support (#965)

* rpc/types: Decode `SubstrateTxStatus` for substrate and smoldot

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Add light client Error

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Add background task to manage RPC responses

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Implement the light client RPC in subxt

Signed-off-by: Alexandru Vasile <[email protected]>

* subxt: Expose light client under experimental feature-flag

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Add development chain spec for local nodes

Signed-off-by: Alexandru Vasile <[email protected]>

* Update cargo lock

Signed-off-by: Alexandru Vasile <[email protected]>

* examples: Add light client example

Signed-off-by: Alexandru Vasile <[email protected]>

* Update sp-* crates and smoldot to use git with branch / rev

Signed-off-by: Alexandru Vasile <[email protected]>

* Apply cargo fmt

Signed-off-by: Alexandru Vasile <[email protected]>

* Fix clippy

Signed-off-by: Alexandru Vasile <[email protected]>

* Import hashmap entry

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Fetch spec only if jsonrpsee feature is enabled

Signed-off-by: Alexandru Vasile <[email protected]>

* Update subxt/src/rpc/lightclient/background.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Fix typo

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Update dev chain spec

Signed-off-by: Alexandru Vasile <[email protected]>

* types: Handle storage replies from chainHead_storage

Signed-off-by: Alexandru Vasile <[email protected]>

* artifacts: Add polkadot spec

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Handle RPC error responses

Signed-off-by: Alexandru Vasile <[email protected]>

* examples: Tx basic with light client for local nodes

Signed-off-by: Alexandru Vasile <[email protected]>

* example: Light client coprehensive example for live chains

Signed-off-by: Alexandru Vasile <[email protected]>

* examples: Remove prior light client example

Signed-off-by: Alexandru Vasile <[email protected]>

* feature: Rename experimental to unstable

Signed-off-by: Alexandru Vasile <[email protected]>

* book: Add light client section

Signed-off-by: Alexandru Vasile <[email protected]>

* testing: Fix clippy

Signed-off-by: Alexandru Vasile <[email protected]>

* lightclient: Ignore validated events

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust tests for light-clients and normal clients

Signed-off-by: Alexandru Vasile <[email protected]>

* testing: Keep lightclient variant

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove support for chainHead_storage for light client

Signed-off-by: Alexandru Vasile <[email protected]>

* Update light client to point to crates.io

Signed-off-by: Alexandru Vasile <[email protected]>

* Update sp-crates from crates.io

Signed-off-by: Alexandru Vasile <[email protected]>

* Replace Atomic with u64

Signed-off-by: Alexandru Vasile <[email protected]>

* Add LightClientBuilder

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust chainspec with provided bootnodes

Signed-off-by: Alexandru Vasile <[email protected]>

* Add potential_relay_chains to light client builder

Signed-off-by: Alexandru Vasile <[email protected]>

* Move the light-client to the background task

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust tracing logs

Signed-off-by: Alexandru Vasile <[email protected]>

* Update book and example

Signed-off-by: Alexandru Vasile <[email protected]>

* Apply cargo fmt

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove dev_spec.json artifact

Signed-off-by: Alexandru Vasile <[email protected]>

* Examples fix duplicate Cargo.toml

Signed-off-by: Alexandru Vasile <[email protected]>

* Use tracing_subscriber crate

Signed-off-by: Alexandru Vasile <[email protected]>

* Fix clippy for different features

Signed-off-by: Alexandru Vasile <[email protected]>

* Add comment about bootNodes

Signed-off-by: Alexandru Vasile <[email protected]>

* Add comment about tracing-sub dependency

Signed-off-by: Alexandru Vasile <[email protected]>

* Run integration-tests with light-client

Signed-off-by: Alexandru Vasile <[email protected]>

* Feature guard some incompatible tests

Signed-off-by: Alexandru Vasile <[email protected]>

* ci: Enable light-client tests under feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* ci: Fix git step name

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust flags for testing

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust warnings

Signed-off-by: Alexandru Vasile <[email protected]>

* Rename feature flag jsonrpsee-ws to jsonrpsee

Signed-off-by: Alexandru Vasile <[email protected]>

* Fix cargo check

Signed-off-by: Alexandru Vasile <[email protected]>

* ci: Run tests on just 2 threads

Signed-off-by: Alexandru Vasile <[email protected]>

* Move light-client to subxt/src/client

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust LightClientBuilder

Signed-off-by: Alexandru Vasile <[email protected]>

* Use ws_url to construct light client for testing

Signed-off-by: Alexandru Vasile <[email protected]>

* Refactor background

Signed-off-by: Alexandru Vasile <[email protected]>

* Address feedback

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove polkadot.spec and trim sub_id

Signed-off-by: Alexandru Vasile <[email protected]>

* Wait for substrate to produce block before connecting light client

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust builder and tests

Signed-off-by: Alexandru Vasile <[email protected]>

* Apply fmt

Signed-off-by: Alexandru Vasile <[email protected]>

* ci: Use release for light client testing

Signed-off-by: Alexandru Vasile <[email protected]>

* Add single test for light-client

Signed-off-by: Alexandru Vasile <[email protected]>

* Wait for more blocks

Signed-off-by: Alexandru Vasile <[email protected]>

* Use polkadot endpoint for testing

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust cargo check

Signed-off-by: Alexandru Vasile <[email protected]>

* examples: Remove light client chain connection example

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust cargo.toml section for the old example

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust background task to use usize for subscription Id

Signed-off-by: Alexandru Vasile <[email protected]>

* Build bootnodes with serde_json::Value directly

Signed-off-by: Alexandru Vasile <[email protected]>

* Make channel between subxt user and subxt background unbounded

Signed-off-by: Alexandru Vasile <[email protected]>

* Update subxt/src/client/lightclient/builder.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Switch to smoldot 0.6.0 from 0.5.0

Signed-off-by: Alexandru Vasile <[email protected]>

* Move testing to `full_client` and `light_client` higher modules

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove subscriptionID type

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove subxt/integration-testing feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust wait_for_blocks documentation

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust utils import for testing

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove into_iter from builder construction

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>

.github/workflows/rust.yml

@@ -166,6 +166,38 @@ jobs:
                   command: nextest
                   args: run --workspace
 
+    nonwasm_light_client_tests:
+        name: "Test Light Client"
+        runs-on: ubuntu-latest-16-cores
+        timeout-minutes: 15
+        steps:
+            - name: Checkout sources
+              uses: actions/checkout@v3
+
+            - name: Download Substrate
+              run: |
+                  curl $SUBSTRATE_URL --output substrate --location
+                  chmod +x substrate
+                  ./substrate --version
+                  mkdir -p ~/.local/bin
+                  mv substrate ~/.local/bin
+
+            - name: Install Rust stable toolchain
+              uses: actions-rs/toolchain@v1
+              with:
+                  profile: minimal
+                  toolchain: stable
+                  override: true
+
+            - name: Rust Cache
+              uses: Swatinem/rust-cache@988c164c3d0e93c4dbab36aaf5bbeb77425b2894 # v2.4.0
+
+            - name: Run tests
+              uses: actions-rs/[email protected]
+              with:
+                  command: test
+                  args: --release --package integration-tests --features unstable-light-client
+
     clippy:
         name: Cargo clippy
         runs-on: ubuntu-latest