Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to select KZG library for PeerDAS #6107

Open
Tracked by #4983
jimmygchen opened this issue Jul 16, 2024 · 2 comments
Open
Tracked by #4983

Add option to select KZG library for PeerDAS #6107

jimmygchen opened this issue Jul 16, 2024 · 2 comments
Labels
blocked das Data Availability Sampling

Comments

@jimmygchen
Copy link
Member

jimmygchen commented Jul 16, 2024

Description

We've recently switched from the c-kzg library to the Rust peerdas-kzg library on the das branch (#5941). The purpose for the switch was to:

  1. Try out a new alternative to c-kzg;
  2. Allow us to merge das branch to unstable without having to update c-kzg to a development version

However we've now lost the option to run the das branch with ckzg, which has been quite stable for a while. We've recently run into a few issues with peerdas-kzg, including slow startup time and stack overflow during block production (#6105). Therefore I think it would be great to have the option to use ckzg library, especially for public denvets, where a small number of reliable supernodes is required.

Ideally we want to be able to:

  1. maintain the ckzg library version for the mainnet code path
  2. optionally include the ckzg peerdas library when it's enabled via a feature flag, so both ckzg library will co-exist when this ckzg-peerdas feature is enabled, however this library will only be referenced by the PeerDAS portion of the code.
  3. allow selecting the ckzg-peerdas library via a hidden runtime flag
@jimmygchen jimmygchen added the das Data Availability Sampling label Jul 16, 2024
@realbigsean realbigsean changed the title Add option to select KZG library for PeerDAS Add option to select KZG library for PeerDAS Dec 17, 2024
@jimmygchen jimmygchen added das Data Availability Sampling and removed das Data Availability Sampling labels Jan 28, 2025
@jimmygchen
Copy link
Member Author

We currently use c-kzg for Deneb, and rust-eth-kzg for PeerDAS. To fully support this we'd need:

  • Re-integrate PeerDAS with c-kzg, so we can have the option to load c-kzg only
  • Use rust-eth-kzg to perform deneb KZG operations, so we can have the option to load rust-eth-kzg only
  • If we complete both changes, then we can have option to choose which KZG library to use, or feature gate them (Add option to select KZG library for PeerDAS #6107)

rust-eth-kzg seems to perform better in PeerDAS - I'm not sure if this has changed.

Alternatively, an option is to maintain the status quo for now, and gradually phase out c-kzg over time? We can't completely switch over to rust KZG yet, AFAIK it hasn't been audited, but I'd assume there are plans to do this.

@jimmygchen
Copy link
Member Author

@pawanjay176 also expressed his preference to maintain the status quo now and eventually get rid of c-kzg. It's unclear whether both libraries will be maintained in the longer term.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked das Data Availability Sampling
Projects
None yet
Development

No branches or pull requests

1 participant