feat(BEDS-536): implement misc fix-internal-txs-from-node cmd with batch processing from Erigon node
#2967
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tion of the block data
…itfly/eth2-beaconchain-explorer into BEDS-536/fix-internal-transfer-v2
Monika-Bitfly
changed the title
fix-internal-txs cmd with batch processing from Erigon nodefix-internal-txs-from-node cmd with batch processing from Erigon node
Tangui-Bitfly
requested changes
Oct 22, 2024
Tangui-Bitfly
approved these changes
Oct 23, 2024
Tangui-Bitfly
pushed a commit
that referenced
this pull request
Jan 22, 2025
…batch processing from Erigon node (#2967) * wip: move itx status parsing logic to transformers func * wip: implementing fix-internal-txs misc cmd * wip: updated geth client initialisation * wip: moved blocks by batch logic to erigon.go * wip: updated block by number batch call * updated blocks batch processing * feat: added transformers for bulk mutations * fix: save current block height in local variable when performing mutation of the block data * fix: add check for start and end block flags * fix: updated erigon client * fix: add check for batchcall size and return if it's empty * cleanup unused code * cleanup * updated cmd name to fix-internal-txs-from-node * fix: uncles, blobGasUsed & excessBlobGas parsing * rm timings, update GetBlocksByBatch func * fix: revised ReindexITxsFromNode func logic * fix: update the logic for parsing uncles, BlobGasUsed and ExcessBlobGas
guybrush
added a commit
that referenced
this pull request
Jan 22, 2025
…tatus (#2965) * fix(internal transaction): recursively revert traces * fix: SQLReaderDb interface missing methods * fix(eth1tx): remove transfers if reverted * fix(internal tx): only save the highest root revert * ci lint * feat: updated eth1 proto file * feat: updated Eth1InternalTransaction in eth1.proto file * feat: added Reverted value to Eth1InternalTransactionIndexed * feat: add check for internal txs while querying the block from erigon node and set the status to 2 if tx has failed partially * feat: updated reverted internal txs check logic * feat(types): add status enum and add status partially executed * refactor(client/erigon): simplify transaction indexing flow * feat(client/erigon): rework geth traces * fix: lint * feat: add status to Eth1TransactionIndexed msg and update hash formatting * fix: updated internal tx handling in GetBlock func * fix: updated internal txs parsing in GetBlock * fix: optimise the memory of internla tx parsing * lint * fix(internal): remove revert + status logic from client to transformers * test(internal): test for revert transformer on tx and itx * feat(cmd): add re-index-blocks cmd * chore(proto): clean proto Eth1Transaction and Eth1TransactionIndexed * fix(TransformItx): empty revertSource + error before skipping * feat(BEDS-536): implement misc `fix-internal-txs-from-node` cmd with batch processing from Erigon node (#2967) * wip: move itx status parsing logic to transformers func * wip: implementing fix-internal-txs misc cmd * wip: updated geth client initialisation * wip: moved blocks by batch logic to erigon.go * wip: updated block by number batch call * updated blocks batch processing * feat: added transformers for bulk mutations * fix: save current block height in local variable when performing mutation of the block data * fix: add check for start and end block flags * fix: updated erigon client * fix: add check for batchcall size and return if it's empty * cleanup unused code * cleanup * updated cmd name to fix-internal-txs-from-node * fix: uncles, blobGasUsed & excessBlobGas parsing * rm timings, update GetBlocksByBatch func * fix: revised ReindexITxsFromNode func logic * fix: update the logic for parsing uncles, BlobGasUsed and ExcessBlobGas * (BEDS-536) Use raw db for resync (#2972) * rpc/erigon: use raw bigtable * rpc/erigon: use cache raw db * rpc/erigon: correct path for geth traces * cmd/reindex: improve performance + fix raw store cache * rpc/erigon: fix sender address * db2/store: add remote server + client * fix ci * store/bigtable: fix range limits go mod * store/bigtable: fix grpc error on close * updated Receipts len check * updated traceMode to geth * fix ci * clenup * rpc/erigon: parse traces geth handle CALLCODE * fix(transform itx): allow internal index == ITX_PER_TX_LIMIT * fix(bigtable): retry on grpc internal err * fix(re index): log error rather than returning error and panicking * feat(re index): re print read error at the end * fix(TransformEnsNameRegistered): return none nil if ignored chainID * fix(db2/WithFallback): fallback on syscall.ECONNRESET * fix(blockHash): prevent wrong calculated hash * fix: return error if mismatch between receipts and transactions length * fix: blockhash read from node response * fix: merge go mod --------- Co-authored-by: Monika-Bitfly <[email protected]> Co-authored-by: Patrick Pfeiffer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes:
fix-internal-txs-from-nodecommand for reindexing internal txs within a specified block rangeeth_getBlockByNumbereth_getBlockReceiptstrace_block