Populate more data in parachain inherents

[kishansagathiya]

Issue summary

• Parachain Inherents have four fields Parent Header, Bitfields, BackedCandidates and Disputes.

• However, we have populated only Parent Header, because that is our minimum requirement for a node to run (build blocks).

• We would like to populate other fields as well.

• The scope of this issue would be find where to and how to get those remaining information (Bitfields, BackedCandidates and Disputes).

• It would most like require a larger parachain support. In which case, figure out what extra feature we might need to populate these data and create other issues accordingly.

Updates after research

This is a large task, one which requires building systems written below

• Collator Protocol
  Receive collations from parachains #3204
• Backing Subsystems
  feat(dot/parachain): Implement Parachain Candidate Backing #3247
• Availability Subsystems
• Approval Subsystems
• Dispute Subsystems
• Some utility Subsystems
• Implement ParachainHost runtime calls
  This can also be done independently Implement ParachainHost runtime calls  #3186

Details of how these subsystems work in polkadot is https://paritytech.github.io/polkadot/book/node/subsystems-and-jobs.html while spec is at https://spec.polkadot.network/#chapter-anv

Basically in order to populate parachain inherents we should be able to get data from collators, that data will make use of all the above mentioned subsystems and in the end we will get parachain inherents for us to put into new blocks.

I feel like the first systems for us to implement should be collator protocol. The Collator Protocol implements the network protocol by which collators and validators communicate. It is used by collators to distribute collations to validators and used by validators to accept collations by collators.

The Collator Protocol is the first in the journey of a parachain block to become part of the relay chain. Thus, we can test this implementation independently. Whereas, all other systems interact with other not-yet-implemented systems, which would require us to mock behaviour.

Other information and links

There is the Node Implementer's Guide. Node implementer explains how parachain host is implemented in Polkadot (rust implementation). It is really helpful because

• you can get the exact definitions of data types,
• description is more detailed than spec
• you can learn an example of software architecture of parachain host

The is also the spec. Personally, I find spec easy to understand. Spec is good to know just the requirement. With node implementer's guide one can get confused on what is the requirement and what is a implementation choice.

[kishansagathiya]

Make sure we can support https://github.com/paritytech/polkadot/releases/tag/v0.9.38
Check paritytech/polkadot#4097 to see if we have to worry about some things.

[kishansagathiya]

Polkadot has implementation of https://paritytech.github.io/polkadot/book/node/overseer.html to create parachain inherents. We would also need this.

I feels like we would increasingly need to add everything that is described in https://paritytech.github.io/polkadot/book/node/index.html. Most of these topics are modules in polkadot codebase.

Polkadot's code uses client and overseer to create parachain inherents. But I bet as I get deeper into it, we would need more things.

overseer is a module that manages different subsystems. It would be a design decision whether we need something like overseer.

There is a subsystem called provisioner which is responsible for choosing a backed candidates and getting inherent data. It looks like we will need some equivalent of provisioner and network subsystem to start things.

Will keep updating this space with my findings.

[kishansagathiya]

@timwu20 @danforbes
I have updated issue description based on my research, I am starting implementation with collator protocol. Let me know if I should create issues for individual subsystems, now or make them as things progress.

[P1sar]

Closing this, and keeping track of all parachain implementation inside the zenhub project