-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ingest historical block hashes #3518
Comments
This would also address #1467 |
I think this is supported for Firehose Blockstreams @maoueh ? |
After #3738 we'll also need to ingest the timestamps to continue supporting the |
We could alternatively lazily fetch blocks when there is a query (similar to #4768 ) |
A note only to point out that this is a pain point on the network. Indexers may be forced to purge the block cache, and there is no easy way to "refill" it. The lazy fetch solution would be great. |
An issue affecting the network, which sends queries by block hash, is that graph node will fail if it cannot find the block hash in its block cache. The block ingestor will ingest blocks within the reorg threshold, but holes can happen and nothing is done to fill them.
Since historical queries mean that all block hashes are potentially necessary, a way to solve this is to run a backfill ingestor job that ingests all historical hashes.
In terms of implementation, we'd want to be able to have only the block hash and number rather than the whole header, which is not valid in the current block cache structure. Also we'd probably want to leverage the firehose and not need a json-rpc implementation of the backfill.
The text was updated successfully, but these errors were encountered: