[ZK Stack] - Understanding Validium mode and changing DB for data-availability

[LimonChillo]

Team or Project

No response

Module Affected

• System contracts (era-system-contracts)
• Smart contracts (era-contracts)
• zkEVM (era-zk_evm)
• Server (zksync-era)
• Prover (era-boojum)
• Fee withdrawer (era-fee-withdrawer)
• Block Explorer (clock-explorer)
• Portal - Wallet + Bridge (dapp-portal)
• Other (please specify below)

Rust Version

1.81.0

Operating System

• Ubuntu
• MacOS
• Windows
• Other (Please specify below)

Issue Description

I'm trying to get a better understanding of the Validium mode. I have selected 'Validium' while creating an ecosystem. (Screenshot 1). And I could confirm this initialisation in the zkStack.yaml (Screenshot 2).

After transmitting some batches from my zk-chain to the L1, I checked the data_availability table (Screenshot 3) I made the following observation:

• A new row was written for every batch committed.
• But inclusion_data for all of these batches are empty.

As per my understanding, when selecting Validium mode, the DA data should be posted to this table? Or is posted elsewhere?

And can I point the DA data to a custom database? When skimming through the code I found 'ObjectStore', could that be helpful? I haven't found any documentation for it.

Thanks a lot everyone! :)

Screenshot 1

Screenshot 2

Screenshot 3

Expected Behavior

DA data to be seen in inclusion_data as above

Current Behavior

inclusion_data is empty as described above

Repository Link (if applicable)

No response

Additional Details

No response

Prior Research

• Yes, I searched for similar issues and did not find any matching ones.

[gleenai[bot]]

It seems you are encountering an issue where the inclusion_data in the data_availability table remains empty despite setting up your ecosystem in Validium mode. This behavior is not expected as the Validium mode should indeed post the Data Availability (DA) data to this table.

Here are a few points to consider and steps you might take:

1. Configuration Check: Ensure that your zkStack.yaml configuration correctly specifies the DA layer and that the DA dispatcher and DA client are properly configured. The DA client should handle the dispatch of blobs and retrieval of inclusion data, while the DA dispatcher manages the periodic fetching and processing of this data.

2. DA Layer Integration: Verify that the integration with your chosen DA layer (e.g., Avail, Celestia, EigenDA) is functioning correctly. This involves checking the network connections and configurations that allow your Validium setup to communicate with the DA layer.

3. Custom Database for DA Data: Regarding your question about pointing DA data to a custom database, the current setup typically uses the data_availability table within the system's predefined database schema. If you are looking to use an 'ObjectStore' or another custom storage solution, this would require significant modifications to the system contracts and possibly the DA client and dispatcher configurations. This is not covered in standard documentation and would likely require custom development.

4. Documentation and Support: Since you mentioned the lack of documentation for 'ObjectStore', it might be beneficial to look into community discussions or reach out for more tailored support. You can visit the ZKsync developers discussions on Github for more in-depth answers and community support.

If these steps do not resolve the issue, it may be necessary to delve deeper into the system logs or seek direct support from the development team to diagnose and address the specific integration issues with the DA layer.

Your feedback helps us serve you better. If this fully addresses your question, please give it a thumbs up. If it doesn't, please give it a thumbs down. This will inform the community and team so we can improve our support.

[sarahschwartz]

Hi @LimonChillo,

As per my understanding, when selecting Validium mode, the DA data should be posted to this table? Or is posted elsewhere?

The data_availability table is used for storing the metadata about the 3rd party DA blobs (more on that here).
The default configuration for a validium is a NoDA validium, which only persists the data in the db (can be found in l1_batches table, pubdata_input field) and doesn't have any metadata about external blobs, which is why the table has empty values.

And can I point the DA data to a custom database? When skimming through the code I found 'ObjectStore', could that be helpful? I haven't found any documentation for it.

You can find info about using a 3rd party DA solution in the FAQs
ObjectStore is a mode in which the pubdata for all batches is stored in a GCS bucket. We will update the docs with more info soon!

[LimonChillo]

Hi @sarahschwartz, thanks so much for your answer, this has been very helpful. I'm looking forward to more docs on custom and 3rd party DA :)

One small suggestion regarding the current docs. Can we update the following paragraph from the FAQs you linked:

The DA related information is stored in the data_availability table in the database, Validium’s new batches can’t be committed if their inclusion data in that table is NULL.

I've assumed the data should be there even in said NoDA Validium.

[LimonChillo]

Hi @sarahschwartz, may I ask a follow up question? :)
When every participant of a Validium ecosystem has access to all of the DA data, what would that mean for privacy?
Is it possible to correlate any practical information from that data? Like any transaction metadata or for example transaction volumes of specific accounts?

[LimonChillo]

I've posted the follow up question as a new thread --> #860