fix(hash): Partially mitigate WASM memory leak #786 using FinalizationRegistry #998
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As described in #786, the
createHash
function currently leaks memory every time it's used: the Hasher instance allocated in the WASM heap is never freed.This change uses
FinalizationRegistry
(which is working correctly on deno@main as of today 🎉) to .free() the Hasher instances in WASM when their correspondingHash
JavaScript objects are garbage collected. This does not entirely resolve the memory leak: the WASM heap will never shrink and release the used memory, but it can significantly mitigate it. In a test on my machine, endlessly callingcreateHash('sha256')
, the WASM heap grew more slowly, and stopped growing around ~70MB, as the allocator inside the WASM was able to reuse the heap space we freed. Without this change, the memory use continued to grow without bound; I stopped it after 1GB.This won't be effective in all cases: The garbage collector may not get a chance to run if you're repeatedly creating hashes in a hot sync loop. If you're passing huge chunks of data into WASM, the heap will still need to be expanded to hold them (discussed in Discord). But I think it's still a significant improvement.
It appears that
wasm-bindgen
does have an option that would allow it to do this automatically internally, but it hasn't been exposed throughwasm-pack
that we're using here, and it's not clear that it will be soon.Test script:
with
added in
hash.ts
to let us check the memory use.