chore: switch to pure-go zstd decoder for snapshot imports

[masih]

Related Issues

• Compare the performane of DataDog/zstd with the pure-go implementation by klauspost/compress #12852

Proposed Changes

Additional Info

The benchmarking performed as part of #12852 shows that the pure-go implementation of zstd decoder is fast-enough for snapshot import.

Switch to it motivated by the desire to reduce CGO dependencies across Lotus.

Fixes #12852

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title conforms with contribution conventions
• Update CHANGELOG.md or signal that this change does not need it per contribution conventions
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[BigLep]

Thanks for the work here @masih . Quick question: why is there a release/backport label? I believe there is no active release train to backport into. I assume the next release (assuming not a rushed security update) will branch from master.

[masih]

why is there a release/backport label? I believe there is no active release train to backport into. I assume the next release (assuming not a rushed security update) will branch from master.

Yep, no need to rush this one. I understand the label to mean "please include in the next release".

Please let me know if I am misinterpreting the label.

[Stebalien]

It means to backport to an existing release candidate. Everything in master will automatically go into the next release unless the release process is already underway.

[masih]

It means to backport to an existing release candidate. Everything in master will automatically go into the next release unless the release process is already underway.

I was explicitly explained to that for anything to end up in a release it needs that label. I am also known to missunderstand things at times.

Thank you for clarifying this for me, Steven.

[Stebalien]

Yeah, I'm guessing the intent of that statement was "if it needs to end up in this release that's currently baking".

[rvagg]

Current lotus to import a mainnet snapshot into a fresh repo:

real    16m47.633s
user    20m44.787s
sys     4m1.878s

Same data, this branch:

real    16m24.879s
user    23m23.562s
sys     4m31.122s

So .. I'm not really sure what to make of the reduction in real time there (I was doing other things during both of these runs so likely skewed) but we can at least say that (1) the pure-go implementation has to work harder to get the job done, and that (2) the bulk of the cost of snapshot import is almost entirely in Badger so zstd effort isn't really a factor here.