[feature request] Add metadata dump subcommand

[dvdplm]

For testing in subxt it would be useful to have a metadata dump subcommand that outputs the metdata to stdout.

Options:
--format/-f: accepts json or hex (default)
--strip-docs: do not include doc strings
--output/-o: output to a file at the given path

Some context

[bkchr]

Substrate itself can not decode the metadata and that would also break when there is a runtime upgrade and the node wasn't upgraded yet.

[dvdplm]

I'd imagine the command would essentially do:

		self.block_or_best(block).map_err(client_err).and_then(|block| {
			self.client
				.runtime_api()
				.metadata(&BlockId::Hash(block))
				.map(Into::into)
				.map_err(|e| Error::Client(Box::new(e)))
		})

Why wouldn't that work?

[bkchr]

You only want the SCALE encoded metadata? I mean that could be fine, but then you also can just call the RPC method.

[dvdplm]

You only want the SCALE encoded metadata? I mean that could be fine, but then you also can just call the RPC method.

Yep. The context here is writing integration tests for subxt and this command would just be a quicker way of getting the metadata used by the tests.
The assumption is that the startup time would be shorter and we wouldn't have to mess with any network calls or even have an rpc client in the test setup. With a metadata dump command we could ensure the right metadata is available in CI rather than using a build.rs step.

[bkchr]

available in CI rather than using a build.rs step.

Can you elaborate this?

About what node do we also actually speak here? The substrate one or Polkadot or what?

[dvdplm]

Can you elaborate this?

The context is this PR where we add a test-runtime crate that queries a substrate node for its metadata and writes it to disk. At the moment it's a plain old --dev --tmp chain but it could be any type of node against whose metadata we want to run tests.

With a metadata dump command we could do the job done in test-runtime's build.rs as a single step in CI instead which is a bit cleaner. It might also be handy while developing, just a quick&easy way to get the metadata rather than running the node and then curl it for the metadata.

Is your concern here that we add too many commands of dubious usefulness?

[bkchr]

I thought about this again and realized that we already have CLI commands optional. So yeah, we could probably provide such a CLI command and chain implementors then can add this if required.

[bkchr]

--format/-f: accepts json or hex (default)
--strip-docs: do not include doc strings
--output/-o: output to a file at the given path

However, we could not support strip docs, because we can not decode the metadata. For format I'm also not sure what you mean with json?

[dvdplm]

However, we could not support strip docs, because we can not decode the metadata. For format I'm also not sure what you mean with json?

Yeah that won't work. I think we might want to provide a way to strip docs in some form at some point – they are only useful for dynamic UIs – but right now we can't implement that. :/

As for the format, that's a brain fart on my end. I was thinking about decoding the metadata.

[shawntabrizi]

Given the nature of Metadata potentially changing block to block, this is a very strange CLI command. I hope it doesnt trick people in the ecosystem to start using this in ways it shouldnt be used.

[bkchr]

Given the nature of Metadata potentially changing block to block

This is something we should prevent. I mean I have removed some of the changing things recently and we should also create a test for this to ensure that metadata doesn't change.

[shawntabrizi]

I mean changing because of runtime upgrades, which could happen all the time depending on the chain.

[bkchr]

I thought about this again and I think this use case should just be done with: https://github.com/chevdor/subwasm

Then you can just point it to a wasm file (instead of a running node) and let it extract metadata from there.

[jsdw]

I thought about this again and I think this use case should just be done with: https://github.com/chevdor/subwasm

Then you can just point it to a wasm file (instead of a running node) and let it extract metadata from there.

A use case for this feature request was that, given some substrate (or compatible) binary, when testing it is useful to be able to easily acquire the metadata currently in use to generate an interface (in this case via subxt), and then run tests against that interface. This replaced a previous approach whereby we hardcoded the metadata used in the tests, which of course could cause test failures against incompatible nodes, and needed manually keeping up to date.

The way we do this currently is to start a substrate node, make the RPC call to acquire metadata and then kill it. This command would simplify that process for us, basically :)

[bkchr]

Yeah I get it, but people would also need to add this cli command to their cli. This stuff isn't coming automatically. But for testing you could also just give it the wasm binary? There isn't that much difference if you provide a node or the runtime directly?