Skip to content
This repository has been archived by the owner on Jan 22, 2025. It is now read-only.

serialize snapshots dir first #25889

Merged
merged 1 commit into from
Jun 13, 2022
Merged

serialize snapshots dir first #25889

merged 1 commit into from
Jun 13, 2022

Conversation

apfitzge
Copy link
Contributor

@apfitzge apfitzge commented Jun 9, 2022

Problem

We can begin indexing our snapshots as we untar other files, however we run into the issue that we don't the know the current_len of the append_vec files until we untar the snapshots directory. Currently the snapshots dir is serialized after the accounts directory.

Summary of Changes

Serialize the snapshots directory before the accounts directory.

Fixes #

@jeffwashington jeffwashington self-requested a review June 13, 2022 13:07
@jeffwashington jeffwashington marked this pull request as ready for review June 13, 2022 13:07
jeffwashington
jeffwashington approved these changes Jun 13, 2022
View reviewed changes
Copy link
Contributor

@jeffwashington jeffwashington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this seems better without negatives I can think of. Attempts to make start from snapshot faster will include using the info in the bank (in snapshots folder) and deferring using the bulk info in accounts.

brooksprumo
brooksprumo approved these changes Jun 13, 2022
View reviewed changes
Copy link
Contributor

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat. lgtm

@jeffwashington
Copy link
Contributor

Added 1.10. We'd like snapshots to be generated like this asap.

@apfitzge apfitzge merged commit 04969a2 into solana-labs:master Jun 13, 2022
@apfitzge apfitzge deleted the feature/serialze_snapshots_first branch June 13, 2022 14:33
@apfitzge apfitzge restored the feature/serialze_snapshots_first branch June 13, 2022 14:35
@apfitzge apfitzge added v1.10 and removed v1.10 labels Jun 13, 2022
mergify bot pushed a commit that referenced this pull request Jun 13, 2022
@apfitzge @mergify
serialize snapshots dir first (#25889)
9f2dbe2 
(cherry picked from commit 04969a2)
@mergify mergify bot mentioned this pull request Jun 13, 2022
Merged
mergify bot added a commit that referenced this pull request Jun 13, 2022
@mergify @apfitzge
serialize snapshots dir first (#25889) (#25938)
f0658a6 
(cherry picked from commit 04969a2)

Co-authored-by: apfitzge <[email protected]>
@apfitzge apfitzge mentioned this pull request Aug 17, 2022
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@brooksprumo brooksprumo brooksprumo approved these changes

@jeffwashington jeffwashington jeffwashington approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@apfitzge @jeffwashington @brooksprumo