Skip to content

Latest commit

 

History

History
149 lines (91 loc) · 6.92 KB

code_migration.md

File metadata and controls

149 lines (91 loc) · 6.92 KB

Code Migration Notes

As we migrate snarkos code into amareleo-chain, we keep notes on key changes. We also create basic code documentation that might help future restructuring.

  • When it comes to describing code, we are not interested in detail. The goal is to sumarize the purpose and identify logically related code.

  • Documented code blocks may cover an entire sub-crate or indvidual modules. Hence we refer to the code using relative paths rather than crate names.

  • No cross-dependencies - This label highlights how the sub-crate/module does not depend on other sub-crates/modules (except for the amareleo-node-metrics sub-crate).

  • The main executable is renamed from snarkos to amareleo-chain.

  • All snarkos sub-crates are renamed from snarkos-* to amareleo-chain-*/amareleo-node-* even when the code is otherwise identical.

  • The amareleo-chain ledger files/folders where renamed as follows:

    snarkos amareleo-chain
    .current-proposal-cache* .amareleo-proposal-cache*
    .ledger-* .amareleo-ledger-*
    /tmp/snakros.log /tmp/amareleo-chain.log

  • snarkos relies on caching the genesis block to disk. The first time a node is run, this will do extra work that speeds-up launching other node instances. The block for testnet is currently stored here:
    /tmp/15983110333109949993277199043008208330432506493933185651419933655310578437field

    amareleo-chain speeds up node startup by including a copy of the genesis block for the different network types as resources. This allows it to quickly load the genesis block from memory as from the first run.

    To appreciate the speed gain make sure to clear any cached blocks and then start snakros and amareleo-chain in any order:

    rm /tmp/*field

  • amareleo-chain uses the --keep-state flag to speed up shutdown. When this flag is not specified, amareleo-chain doesn't perform a clean shutdown, terminating immidiately.


Code Encapsulation Issues

There are some code encapsulation issues within snarkos and snarkvm. Specifically:

  • The minimum committee size requirements are hardcoded in both snarkos and snarkvm. For this reason, at this stage we mock having 4 committee members in a single node. An alternative approach would be to remove/replace the consensus altogether.

  • The ledger directory path is returned by aleo-std | aleo_ledger_dir(). This is invoked directly both from snarkos and snarkvm.

amareleo-chain aims to rename the ledger folder. We achive this by setting StorageMode::Custom with our custom ledger folder. This StorageMode instructs aleo_ledger_dir() to use our custom path.


Notes

  • /node/metrics - Prometheus/Grafana Metrics.
    No cross-dependencies

  • /node/sync/locators - Recent block handling object.
    No cross-dependencies

  • /node/bft/events - Raw encoding/decoding of messages (events) such as batch proposals, batch signatures, etc.
    Dependent on BlockLocators, Metrics

  • /node/bft/src/helpers/channels.rs - Message (events) passing channels communication between threads.
    Dependent on Events, BlockLocators

  • /node/bft/src/helpers/proposal.rs - Proposal object composed of <header, transmissions, signatures>
    No cross-dependencies, except for tests (removed).

  • /node/bft/src/helpers/signed_proposals.rs - A HashMap of recently signed proposals where each maps address => (round, batch_id, signature)
    No cross-dependencies

  • /node/bft/ledger-service - Implements structs exposing the LedgerService trait. The most useful is the CoreLedgerService which encapsulates the LedgerService implementation for validator nodes.

    The struct includes:

    1. snarkvm::Ledger,
    2. Cache of committees organized by round
    3. Latest committee leader
    4. Shutdown flag.

    No cross-dependencies

  • /node/bft/storage-service - Implements memory pool storage structs exposing the StorageService trait. BFTMemoryService provides in-memory storage. BFTPersistentStorage persists to a rocksdb within the ledger folder.
    No cross-dependencies

  • /node/bft/src/helpers/storage.rs - Implements complete memory pool storage in Storage and StorageInner. This includes:

    1. LedgerService
    2. Block Hieght, Round Number
    3. Last garabage collected round.
    4. Maxium number of rounds stored.
    5. Map (round_num => List of (certificate_id, batch_id, signer_addr))
    6. Map (certificate_id => certificate)
    7. Map (batch_id => round_num)
    8. StorageService

    Dependent on LedgerService, StorageService

  • /node/bft/src/helpers/ready.rs - Implements a map of transmissions that have completed processing transmission_id => transmission. Where a transmission can be one of:

    • Ratification
    • Solution
    • Transaction

    No cross-dependencies

  • /node/bft/src/worker.rs - Implements the Worker object responsible for processing transmissions.

    In snarkos this would send out requests/responses for transmissions to peers. In snarkos the receiving side would be the Sync object. The amareleo-chain Worker is stripped down from the network communcation (gateway) component.

    Dependent on LedgerService, StorageService, Ready, Proposal

  • /node/bft/src/sync/mod.rs - Implements the Sync module which is resposnbile to sync the ledger from storage and keeping the ledger in a synced state. In snarkos, Sync receives messages from other peer Worker instances. However in amareleo-chain all that communication was removed. In snarkos, Sync employes BlockSync for exchanging blocks with peers. In amareleo-chain this BlockSync dependency was removed with some of its functianality merged into Sync.

    Dependent on Storage, LedgerService, BlockLocators

  • /node/bft/src/primary.rs - Implements the Primary object, a DAG-based Narwhal mempool manager. This object has been greatly modified to allow a single object to create proposal certificates for itself and other fake validators.

    Dependent on Proposal, Storage, Sync, Worker, LedgerService

  • /node/bft/src/helpers/dag.rs - Implements the DAG object an in-memory DAG for Narwhal to store its rounds.

    No cross-dependencies

  • /node/bft/src/bft.rs - Implements the BFT object bringing together Primary and DAG largely encapsulating Narwhal.

    Dependent on Primary, DAG

  • /node/consensus/src/lib.rs - Implements the Consensus object adding the Bullshark consensus to Narhwal.

    Dependent on BFT, Ledger, Storage

  • /node/rest - Implements rest server.

    Dependent on Consensus