feat(basic_witness_producer_input): Add Basic Witness Producer Input component

[EmilLuta]

What ❔

This PR introduces a new component that will be ran alongside all components in the service. The component extracts all information needed for running Basic Witness Generator in a binary serialized Input and uploads it to Object Store (GCS for now).

Why ❔

Part of work decoupling Prover Subsystems from Server Core. Today, Witness Generators need Server Core Database's access to generate witnesses. More information can be found here for the Witness Generator split and here for the Prover Subsystems and Server core split.

Checklist

• PR title corresponds to the body of PR (we generate changelog entries from PRs).
• Tests for the changes have been added / updated.
• Documentation comments have been added / updated.
• Code has been formatted via zk fmt and zk lint.

[codecov]

Codecov Report

Attention: 145 lines in your changes are missing coverage. Please review.

Comparison is base (ec41692) 35.61% compared to head (090500d) 35.42%.
Report is 11 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #156      +/-   ##
==========================================
- Coverage   35.61%   35.42%   -0.20%     
==========================================
  Files         520      524       +4     
  Lines       28352    28500     +148     
==========================================
- Hits        10098    10096       -2     
- Misses      18254    18404     +150     

Files	Coverage Δ
core/bin/zksync_server/src/main.rs	100.00% <ø> (ø)
core/lib/types/src/block.rs	85.71% <ø> (+0.52%)	⬆️
core/lib/types/src/proofs.rs	40.00% <ø> (-11.03%)	⬇️
...lib/zksync_core/src/metadata_calculator/updater.rs	93.78% <100.00%> (+0.10%)	⬆️
core/lib/zksync_core/src/state_keeper/io/mod.rs	82.25% <ø> (ø)
core/lib/zksync_core/src/state_keeper/keeper.rs	71.86% <ø> (ø)
core/lib/dal/src/lib.rs	27.53% <0.00%> (-0.41%)	⬇️
core/lib/dal/src/time_utils.rs	50.00% <0.00%> (ø)
core/lib/zksync_core/src/metrics.rs	0.00% <0.00%> (ø)
core/lib/state/src/witness.rs	0.00% <0.00%> (ø)
... and 9 more

... and 38 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[RomanBrodetski]

good progress - just left some nits

[popzxc]

I also have some general questions, will ask them separately.

[popzxc]

LGTM except for unwrap.

Well done. This PR was tough -- normally it gets easier later, once you learn what you'll be nitpicked on 😅

[slowli]

Other than small nits, looks fine 👍

[RomanBrodetski]

👍

[spartucus]

Since the current witness_generator use PG instead of blob, as below code:

zksync-era/prover/witness_generator/src/basic_circuits.rs

Line 671 in 1e431a6

let storage = PostgresStorage::new(rt_handle, connection, last_miniblock_number, true);

This Basic Witness Producer component does not work any more, am I understand correctly?

@EmilLuta @shahar4

[RomanBrodetski]

This Basic Witness Producer component does not work any more, am I understand correctly?

@spartucus the component works, but its output is indeed not used. This migration is planned for the coming months - prover subsystem will not have access to any of the core tables. It will use this component output instead

[spartucus]

@RomanBrodetski Thanks for your kind reply.

prover subsystem will not have access to any of the core tables. It will use this component output instead

The BasicWitnessInputProducer's output is WitnessBlockState, is it enough for the prover? Or you will add more data in the output?

For the current code implementation, my question is, given that BasicWitnessInputProducer creates a VmInstance, which executes the transaction and finish the batch, why can't its output be used as a witness? WitnessGenerator on the other hand, uses zkevm_test_harness::external_calls::run_with_fixed_params to generate witnesses, any consideration on this?