Skip to content

Commit

Permalink
Pre-allocate vectors in SSZ decoding (sigp#3417)
Browse files Browse the repository at this point in the history
## Issue Addressed

Fixes a potential regression in memory fragmentation identified by @paulhauner here: sigp#3371 (comment).

## Proposed Changes

Immediately allocate a vector with sufficient size to hold all decoded elements in SSZ decoding. The `size_hint` is derived from the range iterator here:

https://github.com/sigp/lighthouse/blob/2983235650811437b44199f9c94e517e948a1e9b/consensus/ssz/src/decode/impls.rs#L489

## Additional Info

I'd like to test this out on some infra for a substantial duration to see if it affects total fragmentation.
  • Loading branch information
michaelsproul authored and Woodpile37 committed Jan 6, 2024
1 parent d58d458 commit 768f4e0
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions consensus/ssz/src/decode/try_from_iter.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,18 @@ pub trait TryFromIter<T>: Sized {
impl<T> TryFromIter<T> for Vec<T> {
type Error = Infallible;

fn try_from_iter<I>(iter: I) -> Result<Self, Self::Error>
fn try_from_iter<I>(values: I) -> Result<Self, Self::Error>
where
I: IntoIterator<Item = T>,
{
Ok(Self::from_iter(iter))
// Pre-allocate the expected size of the Vec, which is parsed from the SSZ input bytes as
// `num_items`. This length has already been checked to be less than or equal to the type's
// maximum length in `decode_list_of_variable_length_items`.
let iter = values.into_iter();
let (_, opt_max_len) = iter.size_hint();
let mut vec = Vec::with_capacity(opt_max_len.unwrap_or(0));
vec.extend(iter);
Ok(vec)
}
}

Expand Down

0 comments on commit 768f4e0

Please sign in to comment.