mining: Overhaul background template generator.

[davecgh]

Currently, block templates that provide work to PoW miners are generated in response to them calling the getwork RPC. However, unlike in a pure PoW system, in Decred, when a new block is connected, a new block template extending that block can't be generated until the minimum number of required votes for it has been received. This poses a challenge for PoW miners because they are typically unaware of when votes are received. Consequently, miners poll getwork waiting until they see a new template generated when a new block is connected.

Since only a minimum of 3 votes is required to build a template and new templates are only generated in response to the miners calling getwork, as just described, miners often, unfortunately, end up receiving a template that only has 3 votes and begin working on it. Worse, since many new miners are not well versed in the intricacies of Decred voting, they often aren't aware they really need to continue polling for new votes and immediately switch over to the new template with more votes in it in order to avoid receiving a reduced subsidy. This often results in new miners producing blocks with 3 votes.

Moreover, blocks with less than all 5 votes are not only undesirable for the PoW miners because they receive reduced subsidy, they're also undesirable for PoS stakers since it means the votes that were not included don't receive the reward despite having voted simply due to a propagation delay.

Another issue with the existing approach is that since it takes a bit of time to generate and validate templates, particularly as the number of transactions they include rises, miners periodically requesting updated work to include the latest transactions have to wait for the template to be built when they call getwork once the existing cached template is expired. This can result in undesirable delays for them.

In order to address the need to wait for templates to be built in that case, there was recently some preliminary work that implemented a background template generator which asynchronously generates templates in response to events with the intention of allowing access to the latest one without said random delays in the future.

However, that preliminary work did not address the issues around vote propagation such as the tendency for miners to end up with the first block template only containing the 3 fastest-propagating votes, nor did it have robust handling for all potential corner cases and error conditions.

Consequently, this overhauls the background block template generator to add support for intelligent vote propagation handling, handle chain reorganization to alternative blocks in the case the current tip is unable to obtain enough votes, provide a subscription for a stream of template updates, handle template generation errors, consider the synchronization state as a part of determining if the chain is current, track the reason for template updates, block current template retrieval during operations that are in the process of making it stale, and correct several corner cases.

It should be noted that this only implements the infrastructure and does not switch getwork or the CPU miner over to use the background templates yet. This will be done in future commits.

The following is a high-level overview of the semantics this implements:

• Generate new templates immediately when prior to stake validation height since no votes are required
• Do not generate templates for intermediate blocks during a chain reorganization
• Do not generate templates before the chain is considered synchronized
• Prefer to create templates with maximum votes through the use of a timeout once the minimum votes have been received to provide the votes an opportunity to propagate with a fallback for immediate generation as soon as all votes are received
  • In the case the timeout expires and a template is created with less than the maximum number of votes, generate a new template immediately upon receiving more votes for the block it extends
• In the event there are competing blocks at the current tip, prefer to build a template on the first seen block so long as it receives the minimum number of required votes within a few seconds
• Generate new templates immediately when a block is disconnected to support future planned chain invalidation capabilities
• Generate new templates periodically when there are new regular transactions to include
• Schedule retries in the rare event template generation fails
• Allow clients to subscribe for updates every time a new template is successfully generated along with a reason why it was generated
• Provide direct access to the most-recently generated template
  • Block while generating new templates that will make the current template stale (e.g. new parent or new votes)

The following is a high-level overview of the changes:

• Initialize PRNG once instead of at every invocation
• Implement an asynchronous queue for all events to ensure normal chain and vote processing is not blocked
• Provide ability to subscribe for all template updates over a single channel to ensure none can indadvertently be missed as is possible with the current design
  • Always deliver existing template upon registration
  • Drop templates if the receiver falls too far behind in the same way tickers work
• Introduce tracking the reason for template updates so that it can ultimately be presented to miners to allow them to make better decisions about when to force their workers to switch
• Consider the current network sync state when checking if the chain is current
• Introduce a regen handler state such as timeouts, blocks to monitor for votes, and the block the next template should extend
• Add logic for selectively reacting to votes on the block the current template is extending, the current tip, and alternate competing blocks at the current tip using the aforementioned semantics
• Perform all template generation in separate goroutines with a cancellable context
  • Cancel any in progress templates that are being generated whenever generating a new one
• Introduce blocking current template retrieval when a new template that would cause the existing one to become stale is being generated
• Modify periodic regen handling to use a resettable timer for better efficiency and more fine grained control
• Remove template pool as that is something that should be handled by the code that is actually handing the templates out
• Rename and export the event notification funcs to make it clear they're not internal functions and also make it easier to eventually move the code into the mining package
• Expand and improve comments
• Store and return template generation errors
  • Schedule retry in the case of failure
• Correct several cases that were not being handled correctly and some undesirable behaviors such as block disconnects (as opposed to reorgs), reorganization to side chains, and notifying with stale templates

[dajohi]

testnet3 miner tOK for a week+