Header Forward Compatibility

[MiguelDD1]

Description

Header has been split into 2 parts:

• A common header which is similar to the generic substrate header,
• And a header extension field which supports forward compatibility using versions.

Other changes

• Data Root build uses beefy_merkle_tree and supports no-std.

[linear]

AVA-397 Compatibility issue between 1.0 & 1.1

Header changes in Header (at KateCommitment) makes incompatibilities between 1.0 and 1.1.

I solved the issue of the header de-codification, but v1.1 is still unable to synchronize with a 1.0 network. It looks the header codification is hight coupled with other stuff like sc_consensus_baba crates, as you can see in the following crash backtrace:

2022-10-11 12:07:51 Polygon Avail Node    
2022-10-11 12:07:51 ✌️  version 1.1.0-062d946-x86_64-linux-gnu    
2022-10-11 12:07:51 ❤️  by Anonymous, 2017-2022    
2022-10-11 12:07:51 📋 Chain specification: Avail-Dev    
2022-10-11 12:07:51 🏷 Node name: Charlie    
2022-10-11 12:07:51 👤 Role: AUTHORITY    
2022-10-11 12:07:51 💾 Database: RocksDb at /mnt/storage/home/miguel/polygon/avail_1/base/charlie/chains/Dev/db/full    
2022-10-11 12:07:51 ⛓  Native runtime: data-avail-5 (data-avail-0.tx1.au10)    

====================

Version: 1.1.0-062d946-x86_64-linux-gnu

0: sp_panic_handler::panic_hook
at /home/miguel/.cargo/git/checkouts/substrate-7e08433d4c370a21/afb74de/primitives/panic-handler/src/lib.rs:166:18
1: sp_panic_handler::set::{{closure}}
at /home/miguel/.cargo/git/checkouts/substrate-7e08433d4c370a21/afb74de/primitives/panic-handler/src/lib.rs:62:12
2: std::panicking::rust_panic_with_hook
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/std/src/panicking.rs:610:17
3: std::panicking::begin_panic_handler::{{closure}}
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/std/src/panicking.rs:502:13
4: std::sys_common::backtrace::__rust_end_short_backtrace
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/std/src/sys_common/backtrace.rs:139:18
5: rust_begin_unwind
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/std/src/panicking.rs:498:5
6: core::panicking::panic_fmt
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/core/src/panicking.rs:107:14
7: core::panicking::panic_display
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/core/src/panicking.rs:63:5
8: core::option::expect_failed
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/core/src/option.rs:1663:5
9: core::option::Option<T>::expect
at /rustc/efec545293b9263be9edfb283a7aa66350b3acbf/library/core/src/option.rs:710:21
10: sc_consensus_babe::prune_finalized
at /home/miguel/.cargo/git/checkouts/substrate-7e08433d4c370a21/afb74de/client/consensus/babe/src/lib.rs:1659:26
11: sc_consensus_babe::block_import
at /home/miguel/.cargo/git/checkouts/substrate-7e08433d4c370a21/afb74de/client/consensus/babe/src/lib.rs:1702:2
12: data_avail::service::new_partial
at node/src/service.rs:237:34
13: data_avail::service::new_full_base
at node/src/service.rs:364:6
14: data_avail::service::new_full
at node/src/service.rs:578:2

In order to prevent future incompatibilities, we are going to add the version to the header, i.e.:

mod v1;
mod v2;
/// ...

pub enum Header<N, H> {
	V1(v1::Header<N,H>),
    // Future versions ...
	V2(v2::Header<N,H>),
    ...
	Unknown,
}

[codecov-commenter]

Codecov Report

Base: 50.80% // Head: 50.79% // Decreases project coverage by -0.00% ⚠️

Coverage data is based on head (b314047) compared to base (d6d79cf).
Patch coverage: 86.79% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop      #99      +/-   ##
===========================================
- Coverage    50.80%   50.79%   -0.01%     
===========================================
  Files           50       52       +2     
  Lines         9035     8943      -92     
===========================================
- Hits          4590     4543      -47     
+ Misses        4445     4400      -45     

Impacted Files	Coverage Δ
node/src/chain_spec.rs	0.00% <0.00%> (ø)
pallets/mocked_runtime/src/test_xt.rs	69.44% <ø> (ø)
pallets/nomad/da-bridge/src/mock.rs	62.66% <0.00%> (-0.85%)	⬇️
pallets/nomad/home/src/mock.rs	75.75% <0.00%> (-1.17%)	⬇️
pallets/nomad/updater-manager/src/mock.rs	56.75% <0.00%> (-1.58%)	⬇️
pallets/system/benchmarking/src/mock.rs	50.00% <0.00%> (-3.34%)	⬇️
pallets/system/src/mock.rs	78.94% <0.00%> (-2.14%)	⬇️
rpc/kate-rpc/src/lib.rs	0.57% <ø> (ø)
runtime/src/lib.rs	9.23% <31.81%> (+0.92%)	⬆️
pallets/system/src/header_builder.rs	84.41% <77.41%> (+4.58%)	⬆️
... and 22 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.