Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

batch-processor: support hot blocks #344

Closed
wants to merge 43 commits into from
Closed

Conversation

belopash
Copy link
Contributor

No description provided.

@belopash belopash changed the base branch from master to tron September 29, 2024 15:42
@belopash belopash marked this pull request as ready for review September 29, 2024 20:57
@belopash
Copy link
Contributor Author

will fix other streams after review

let state = await this.getDatabaseState()

// remove all hot block to start from the finalized head
state = {...state, top: []}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not polite to loose progress.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not polite to loose progress.

Maybe, but it's just easier to do it in this way, rather than check all "top" blocks

let chainHeight = await this.chainHeight.get()
private async processBatch(prevState: HotDatabaseState, finalizedHead: HashAndHeight, blocks: B[]): Promise<HotDatabaseState> {
assert(
prevState.height < finalizedHead.height ||
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finalized head reported by the data source might lie below the previously finalized head at the start of processing.

Copy link
Contributor Author

@belopash belopash Oct 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finalized head reported by the data source might lie below the previously finalized head at the start of processing.

How is it possible? It should be considered as an error


let mappingStartTime = process.hrtime.bigint()
let prevHead = maybeLast(prevState.top) || prevState
let nextHead = maybeLast(nextState.top) || nextState
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

State tracking procedure above feels tricky, but most importantly it is not need (both for the current and future database interfaces).

let baseHead = getParent(blocks[0])
let nextHead = last(blocks)

Copy link
Contributor Author

@belopash belopash Oct 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Processor doesn't know anything about parent blocks. Parant hash is not a part of the DataSource interface

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I've already said, from the processor perpective it's better to have explicit "rollbacks" array in sream response

Base automatically changed from tron to master October 7, 2024 08:43
@belopash belopash closed this Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants