Add option for automatic metadata download to macro

[cmichi]

Problem Statement

For compile-time generated types subxt currently requires having the metadata as a fixture locally:

#[subxt::subxt(runtime_metadata_path = "metadata/contracts-node.scale")]
pub mod api {}

We are using subxt as part of an E2E testing framework that we'll ship with the next major ink! release. Developers can then write tests for their contracts in the usual #[test] style, with an E2E test being executed in the background.

As ink! needs to support all kind of Substrate runtimes this would currently require us to make subxt-cli a dep of ink!, and devs would have to execute subxt metadata > contracts-node.scale before running their E2E tests.

Solution Ideas

Could we make the runtime_metadata_path optional and if omitted download the metadata from the node for each macro invocation?

[niklasad1]

or have something else like:

#[subxt::subxt(fetch_metadata_url = "http://myurl.com", name = "chain_foo")]
pub mod api {}

name would store the metadata as chain_foo.scale, not exactly sure how to deal with duplicate metadata files and to avoid downloading already fetched metadata

[jsdw]

I guess there's no reason it'd need to store the metadata either; just download it and pass it straight into the codegen stuff. it would mean you'd have to have a node available to download from, but there's precedent in things like sqlx which requires compile time access to a running DB for instance (well, by default).

[cmichi]

I guess there's no reason it'd need to store the metadata either; just download it and pass it straight into the codegen stuff. it would mean you'd have to have a node available to download from, but there's precedent in things like sqlx which requires compile time access to a running DB for instance (well, by default).

This is the solution that would work best for us.

The context is that having a fixed file path typically leads to all kinds of problems if the contract is built from a different folder or in a CI setup, where the target dir is often dependent on the environment variables.

[jsdw]

That makes sense!

So I guess since we already have:

#[subxt::subxt(runtime_metadata_path = "metadata/contracts-node.scale")]
pub mod api {}

I'd propose we also support something like:

#[subxt::subxt(runtime_metadata_url = "ws://localhost:9944")]
pub mod api {}

(supporting http, ws, and probably also https and wss to maximise the locations you can acquire it from)

[cmichi]

That would be perfect!

[ascjones]

This option should also be available in the subxt-codegen api

subxt/codegen/src/api/mod.rs

Line 54 in e48f0e3

pub fn generate_runtime_api<P>(

Both the macro and the CLI call in to this, and possibly could just move some of the code from the CLI which already fetches the remove metadata into the codegen.

Edit here is where the CLI calls into the codegen:

subxt/cli/src/main.rs

Line 313 in c55b7ba

let runtime_api = generator.generate_runtime(item_mod, derives);