Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce validator client #7854

Merged
merged 44 commits into from
Aug 16, 2024
Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
78dbe1b
feat(vc): introduce validator client
Maddiaa0 Aug 1, 2024
34b32af
temp
Maddiaa0 Aug 5, 2024
59b82a6
move some stuff around
Maddiaa0 Aug 6, 2024
f3fe10b
feat: memory attestation pull
Maddiaa0 Aug 7, 2024
3e95501
chore: add gossipable to cpell
Maddiaa0 Aug 7, 2024
2c0f307
chore: introduce block with attestations structure
Maddiaa0 Aug 7, 2024
0f515e5
chore: update validotr service
Maddiaa0 Aug 7, 2024
b2f78ed
feat: light integration with sequencer
Maddiaa0 Aug 7, 2024
8831381
fmt
Maddiaa0 Aug 7, 2024
b7a5131
fmt
Maddiaa0 Aug 7, 2024
8b3c55e
feat: start p2p test rework
Maddiaa0 Aug 7, 2024
6fa1fc1
temp: change attestation map
Maddiaa0 Aug 8, 2024
437635a
feat: test that requires collected attestations to produce block
Maddiaa0 Aug 8, 2024
5d2808b
fixes: fmt + types
Maddiaa0 Aug 8, 2024
0c50ee9
fix: header
Maddiaa0 Aug 8, 2024
4c0a943
Merge branch 'master' into md/08-01-feat_vc_introduce_validator_client
Maddiaa0 Aug 8, 2024
39f5d90
fixes
Maddiaa0 Aug 8, 2024
c8f1dba
Merge branch 'master' into md/08-01-feat_vc_introduce_validator_client
Maddiaa0 Aug 9, 2024
45ac399
chore: rename BaseHashType to Buffer32
Maddiaa0 Aug 12, 2024
a1ff397
chore: signature type rework
Maddiaa0 Aug 12, 2024
33fb87f
fix: viem types
Maddiaa0 Aug 12, 2024
759982b
f,t
Maddiaa0 Aug 12, 2024
d2f0854
fix: sign over archive
Maddiaa0 Aug 13, 2024
0cc6920
sweep
Maddiaa0 Aug 13, 2024
1a0d7e8
Merge branch 'master' into md/08-01-feat_vc_introduce_validator_client
Maddiaa0 Aug 13, 2024
07c4203
chore: rm attestations test + clean
Maddiaa0 Aug 13, 2024
064db45
forge fmt
Maddiaa0 Aug 13, 2024
95dfabf
chore: sweep + annotate issues
Maddiaa0 Aug 14, 2024
842d8ce
fmt
Maddiaa0 Aug 14, 2024
27b33c5
toggle is devnet
Maddiaa0 Aug 14, 2024
e9a32d2
fix: update constants
Maddiaa0 Aug 14, 2024
c284de0
fix: yarn prepare
Maddiaa0 Aug 14, 2024
27ad8bb
fix: yarn config
Maddiaa0 Aug 14, 2024
f5b45b6
fix: webpack
Maddiaa0 Aug 14, 2024
f3adc08
Merge branch 'master' into md/08-01-feat_vc_introduce_validator_client
Maddiaa0 Aug 14, 2024
2c1aeb7
fix: tests are coupled apparently
Maddiaa0 Aug 14, 2024
0e25b4e
fix: revert hash caching on header :(
Maddiaa0 Aug 14, 2024
c4614fc
fix: add default key for validator client in sandbox
Maddiaa0 Aug 14, 2024
8ee6efc
fix: fmt
Maddiaa0 Aug 14, 2024
8986e68
fmt
Maddiaa0 Aug 14, 2024
eefca64
Merge branch 'master' into md/08-01-feat_vc_introduce_validator_client
Maddiaa0 Aug 14, 2024
dd80451
review fixes
Maddiaa0 Aug 16, 2024
bde2469
Merge branch 'master' into md/08-01-feat_vc_introduce_validator_client
Maddiaa0 Aug 16, 2024
5a47571
fix: constants value wrong
Maddiaa0 Aug 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@
"fuzzers",
"gitmodules",
"gitrepo",
"Gossipable",
"gossipsub",
"grumpkin",
"gtest",
Expand Down
1 change: 1 addition & 0 deletions docker-compose.provernet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ services:
SEQ_MIN_SECONDS_BETWEEN_BLOCKS: 0
SEQ_RETRY_INTERVAL: 10000
SEQ_PUBLISHER_PRIVATE_KEY: "0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a"
VALIDATOR_PRIVATE_KEY: "0x47e179ec197488593b187f80a00eb0da91f1b9d0b13f8733639f19c30a34926a"
PROVER_REAL_PROOFS: "${PROVER_REAL_PROOFS:-false}"
ASSUME_PROVEN_UNTIL_BLOCK_NUMBER: "${ASSUME_PROVEN_UNTIL_BLOCK_NUMBER:-4}"
P2P_ENABLED: false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,9 @@ SEQ_MAX_TX_PER_BLOCK=32 # Maximum txs to go on a block. (default: 32)
SEQ_MIN_TX_PER_BLOCK=1 # Minimum txs to go on a block. (default: 1)
SEQ_MAX_SECONDS_BETWEEN_BLOCKS=0 # Sequencer will produce a block with less than the min number of txs once this threshold is reached. (default: 0, means disabled)
SEQ_MIN_SECONDS_BETWEEN_BLOCKS=0 # Minimum seconds to wait between consecutive blocks. (default: 0)

## Validator variables ##
VALIDATOR_PRIVATE_KEY=0x01234567890abcde01234567890abcde # Private key of the ethereum account that will be used to perform validator duties
```

**PXE**
Expand Down
3 changes: 3 additions & 0 deletions l1-contracts/src/core/sequencer_selection/ILeonidas.sol
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ interface ILeonidas {
function getTimestampForSlot(uint256 _slotNumber) external view returns (uint256);

// Likely removal of these to replace with a size and indiviual getter
// Note: This function is not view - as it makes a call to setupEpoch under the hood, but it should only be
// used as a view function
function getCurrentEpochCommittee() external returns (address[] memory);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See below.

function getEpochCommittee(uint256 _epoch) external view returns (address[] memory);
function getValidators() external view returns (address[] memory);
}
12 changes: 12 additions & 0 deletions l1-contracts/src/core/sequencer_selection/Leonidas.sol
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,18 @@ contract Leonidas is Ownable, ILeonidas {
return epochs[_epoch].committee;
}

/**
* @notice Get the validator set for the current epoch
*
* @dev Makes a call to setupEpoch under the hood, this should ONLY be called as a view function, and not from within
* this contract.
* @return The validator set for the current epoch
*/
function getCurrentEpochCommittee() external override(ILeonidas) returns (address[] memory) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this, I think we could be a bit smarter just to avoid the issue.

In the getProposerAt we end up drawing a committee if there is not one already. Think we could likely just make a version of that which just don't draw from the set but just returns it 🤷 Seems slightly better for us to not have the option where someone ends up calling it directly. Something I have done in the past have been having an extension to a contract with extra view functions that is not necessarily used within itself, essentially separating the external views 🤷.

We could potentially just alter the `getEpochCommittee(uint256 _epoch)

function getCommitteeAt(uint256 _ts) public view override(ILeonidas) returns (address) {
    uint256 epochNumber = getEpochAt(_ts);
    uint256 slot = getSlotAt(_ts);
    if (epochNumber == 0) {
      return new address[1]();
    }

    Epoch storage epoch = epochs[epochNumber];

    if (epoch.sampleSeed != 0) {
      uint256 committeeSize = epoch.committee.length;
      if (committeeSize == 0) {
        return new address[1]();
      }
      return epoch.committee;
    }

    // Allow anyone if there is no validator set
    if (validatorSet.length() == 0) {
      return new address[1]();
    }

    // Emulate a sampling of the validators
    uint256 sampleSeed = _getSampleSeed(epochNumber);
    return _sampleValidators(epochNumber, sampleSeed);
  }

setupEpoch();
return epochs[getCurrentEpoch()].committee;
}

/**
* @notice Get the validator set
*
Expand Down
1 change: 1 addition & 0 deletions yarn-project/aztec-node/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
"@aztec/simulator": "workspace:^",
"@aztec/telemetry-client": "workspace:^",
"@aztec/types": "workspace:^",
"@aztec/validator-client": "workspace:^",
"@aztec/world-state": "workspace:^",
"koa": "^2.14.2",
"koa-router": "^12.0.0",
Expand Down
11 changes: 11 additions & 0 deletions yarn-project/aztec-node/src/aztec-node/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { type ConfigMappingsType, booleanConfigHelper, getConfigFromMappings } f
import { type P2PConfig, p2pConfigMappings } from '@aztec/p2p';
import { type ProverClientConfig, proverClientConfigMappings } from '@aztec/prover-client';
import { type SequencerClientConfig, sequencerClientConfigMappings } from '@aztec/sequencer-client';
import { type ValidatorClientConfig, validatorClientConfigMappings } from '@aztec/validator-client';
import { type WorldStateConfig, worldStateConfigMappings } from '@aztec/world-state';

import { readFileSync } from 'fs';
Expand All @@ -16,17 +17,22 @@ export { sequencerClientConfigMappings, SequencerClientConfig } from '@aztec/seq
*/
export type AztecNodeConfig = ArchiverConfig &
SequencerClientConfig &
ValidatorClientConfig &
ProverClientConfig &
WorldStateConfig &
Pick<ProverClientConfig, 'bbBinaryPath' | 'bbWorkingDirectory' | 'realProofs'> &
P2PConfig & {
/** Whether the sequencer is disabled for this node. */
disableSequencer: boolean;

/** Whether the validator is disabled for this node */
disableValidator: boolean;
};

export const aztecNodeConfigMappings: ConfigMappingsType<AztecNodeConfig> = {
...archiverConfigMappings,
...sequencerClientConfigMappings,
...validatorClientConfigMappings,
...proverClientConfigMappings,
...worldStateConfigMappings,
...p2pConfigMappings,
Expand All @@ -35,6 +41,11 @@ export const aztecNodeConfigMappings: ConfigMappingsType<AztecNodeConfig> = {
description: 'Whether the sequencer is disabled for this node.',
...booleanConfigHelper(),
},
disableValidator: {
env: 'VALIDATOR_DISABLED',
description: 'Whether the validator is disabled for this node.',
...booleanConfigHelper(),
},
};

/**
Expand Down
4 changes: 2 additions & 2 deletions yarn-project/aztec-node/src/aztec-node/http_rpc_server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ import {
import { FunctionSelector, Header } from '@aztec/circuits.js';
import { NoteSelector } from '@aztec/foundation/abi';
import { AztecAddress } from '@aztec/foundation/aztec-address';
import { Buffer32 } from '@aztec/foundation/buffer';
import { EthAddress } from '@aztec/foundation/eth-address';
import { Fr } from '@aztec/foundation/fields';
import { BaseHashType } from '@aztec/foundation/hash';
import { JsonRpcServer } from '@aztec/foundation/json-rpc/server';

/**
Expand All @@ -41,7 +41,7 @@ export function createAztecNodeRpcServer(node: AztecNode) {
TxEffect,
LogId,
TxHash,
BaseHashType,
Buffer32,
PublicDataWitness,
SiblingPath,
},
Expand Down
14 changes: 12 additions & 2 deletions yarn-project/aztec-node/src/aztec-node/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ import { Timer } from '@aztec/foundation/timer';
import { type AztecKVStore } from '@aztec/kv-store';
import { createStore, openTmpStore } from '@aztec/kv-store/utils';
import { SHA256Trunc, StandardTree, UnbalancedTree } from '@aztec/merkle-tree';
import { AztecKVTxPool, type P2P, createP2PClient } from '@aztec/p2p';
import { AztecKVTxPool, InMemoryAttestationPool, type P2P, createP2PClient } from '@aztec/p2p';
import { getCanonicalClassRegisterer } from '@aztec/protocol-contracts/class-registerer';
import { getCanonicalFeeJuice } from '@aztec/protocol-contracts/fee-juice';
import { getCanonicalInstanceDeployer } from '@aztec/protocol-contracts/instance-deployer';
Expand All @@ -74,6 +74,7 @@ import {
type ContractInstanceWithAddress,
type ProtocolContractAddresses,
} from '@aztec/types/contracts';
import { createValidatorClient } from '@aztec/validator-client';
import { MerkleTrees, type WorldStateSynchronizer, createWorldStateSynchronizer } from '@aztec/world-state';

import { type AztecNodeConfig, getPackageInfo } from './config.js';
Expand Down Expand Up @@ -149,7 +150,13 @@ export class AztecNodeService implements AztecNode {
config.transactionProtocol = `/aztec/tx/${config.l1Contracts.rollupAddress.toString()}`;

// create the tx pool and the p2p client, which will need the l2 block source
const p2pClient = await createP2PClient(config, store, new AztecKVTxPool(store, telemetry), archiver);
const p2pClient = await createP2PClient(
config,
store,
new AztecKVTxPool(store, telemetry),
new InMemoryAttestationPool(),
archiver,
);

// now create the merkle trees and the world state synchronizer
const worldStateSynchronizer = await createWorldStateSynchronizer(config, store, archiver);
Expand All @@ -166,11 +173,14 @@ export class AztecNodeService implements AztecNode {

const simulationProvider = await createSimulationProvider(config, log);

const validatorClient = createValidatorClient(config, p2pClient);

// now create the sequencer
const sequencer = config.disableSequencer
? undefined
: await SequencerClient.new(
config,
validatorClient,
p2pClient,
worldStateSynchronizer,
archiver,
Expand Down
3 changes: 3 additions & 0 deletions yarn-project/aztec-node/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@
{
"path": "../types"
},
{
"path": "../validator-client"
},
{
"path": "../world-state"
}
Expand Down
4 changes: 2 additions & 2 deletions yarn-project/aztec.js/src/rpc_clients/pxe_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import {
Point,
} from '@aztec/circuits.js';
import { NoteSelector } from '@aztec/foundation/abi';
import { BaseHashType } from '@aztec/foundation/hash';
import { Buffer32 } from '@aztec/foundation/buffer';
import { createJsonRpcClient, makeFetch } from '@aztec/foundation/json-rpc/client';

/**
Expand Down Expand Up @@ -57,7 +57,7 @@ export const createPXEClient = (url: string, fetch = makeFetch([1, 2, 3], false)
Point,
TxExecutionRequest,
TxHash,
BaseHashType,
Buffer32,
},
{
EncryptedNoteL2BlockL2Logs,
Expand Down
4 changes: 4 additions & 0 deletions yarn-project/aztec/src/sandbox.ts
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,10 @@ export async function createSandbox(config: Partial<SandboxConfig> = {}) {
const privKey = hdAccount.getHdKey().privateKey;
aztecNodeConfig.publisherPrivateKey = `0x${Buffer.from(privKey!).toString('hex')}`;
}
if (!aztecNodeConfig.validatorPrivateKey || aztecNodeConfig.validatorPrivateKey === NULL_KEY) {
const privKey = hdAccount.getHdKey().privateKey;
aztecNodeConfig.validatorPrivateKey = `0x${Buffer.from(privKey!).toString('hex')}`;
}

if (!aztecNodeConfig.p2pEnabled) {
await deployContractsToL1(aztecNodeConfig, hdAccount);
Expand Down
4 changes: 4 additions & 0 deletions yarn-project/aztec/terraform/node/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,10 @@ resource "aws_ecs_task_definition" "aztec-node" {
name = "SEQ_PUBLISHER_PRIVATE_KEY"
value = local.sequencer_private_keys[count.index]
},
{
name = "VALIDATOR_PRIVATE_KEY"
value = local.sequencer_private_keys[count.index]
},
{
name = "ROLLUP_CONTRACT_ADDRESS"
value = data.terraform_remote_state.l1_contracts.outputs.rollup_contract_address
Expand Down
3 changes: 2 additions & 1 deletion yarn-project/circuit-types/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@
"jest": "^29.5.0",
"jest-mock-extended": "^3.0.3",
"ts-node": "^10.9.1",
"typescript": "^5.0.4"
"typescript": "^5.0.4",
"viem": "^2.7.15"
},
"files": [
"dest",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { FunctionSelector, Header } from '@aztec/circuits.js';
import { EventSelector, NoteSelector } from '@aztec/foundation/abi';
import { AztecAddress } from '@aztec/foundation/aztec-address';
import { Buffer32 } from '@aztec/foundation/buffer';
import { EthAddress } from '@aztec/foundation/eth-address';
import { Fr } from '@aztec/foundation/fields';
import { BaseHashType } from '@aztec/foundation/hash';
import { createJsonRpcClient, defaultFetch } from '@aztec/foundation/json-rpc/client';

import { type AztecNode } from '../../interfaces/aztec-node.js';
Expand Down Expand Up @@ -41,7 +41,7 @@ export function createAztecNodeClient(url: string, fetch = defaultFetch): AztecN
TxEffect,
LogId,
TxHash,
BaseHashType,
Buffer32,
PublicDataWitness,
SiblingPath,
},
Expand Down
28 changes: 17 additions & 11 deletions yarn-project/circuit-types/src/p2p/block_attestation.test.ts
Original file line number Diff line number Diff line change
@@ -1,22 +1,28 @@
// Serde test for the block attestation type
import { makeHeader } from '@aztec/circuits.js/testing';

import { BlockAttestation } from './block_attestation.js';

const makeBlockAttestation = (): BlockAttestation => {
const blockHeader = makeHeader(1);
const signature = Buffer.alloc(64, 1);

return new BlockAttestation(blockHeader, signature);
};
import { makeBlockAttestation, randomSigner } from './mocks.js';

describe('Block Attestation serialization / deserialization', () => {
it('Should serialize / deserialize', () => {
const attestation = makeBlockAttestation();
it('Should serialize / deserialize', async () => {
const attestation = await makeBlockAttestation();

const serialized = attestation.toBuffer();
const deserialized = BlockAttestation.fromBuffer(serialized);

expect(deserialized).toEqual(attestation);
});

it('Should serialize / deserialize + recover sender', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🫡

const account = randomSigner();

const proposal = await makeBlockAttestation(account);
const serialized = proposal.toBuffer();
const deserialized = BlockAttestation.fromBuffer(serialized);

expect(deserialized).toEqual(proposal);

// Recover signature
const sender = await deserialized.getSender();
expect(sender.toChecksumString()).toEqual(account.address);
});
});
47 changes: 39 additions & 8 deletions yarn-project/circuit-types/src/p2p/block_attestation.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
import { Header } from '@aztec/circuits.js';
import { BaseHashType } from '@aztec/foundation/hash';
import { EthAddress, Header } from '@aztec/circuits.js';
import { Buffer32 } from '@aztec/foundation/buffer';
import { Fr } from '@aztec/foundation/fields';
import { BufferReader, serializeToBuffer } from '@aztec/foundation/serialize';

import { recoverMessageAddress } from 'viem';

import { Gossipable } from './gossipable.js';
import { Signature } from './signature.js';
import { TopicType, createTopicString } from './topic_type.js';

export class BlockAttestationHash extends BaseHashType {
export class BlockAttestationHash extends Buffer32 {
constructor(hash: Buffer) {
super(hash);
}
Expand All @@ -20,11 +24,15 @@ export class BlockAttestationHash extends BaseHashType {
export class BlockAttestation extends Gossipable {
static override p2pTopic: string;

private sender: EthAddress | undefined;

constructor(
/** The block header the attestation is made over */
public readonly header: Header,
// TODO: temp?
public readonly archive: Fr,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Temp?

Teasing aside. Yes think we will move it to the header hash as discussed in #7727 (comment)

/** The signature of the block attester */
public readonly signature: Buffer,
public readonly signature: Signature,
) {
super();
}
Expand All @@ -33,16 +41,39 @@ export class BlockAttestation extends Gossipable {
this.p2pTopic = createTopicString(TopicType.block_attestation);
}

override p2pMessageIdentifier(): BaseHashType {
return BlockAttestationHash.fromField(this.header.hash());
override p2pMessageIdentifier(): Buffer32 {
return BlockAttestationHash.fromField(this.archive);
}

/**Get sender
*
* Lazily evaluate and cache the sender of the attestation
* @returns The sender of the attestation
*/
async getSender() {
if (!this.sender) {
// Recover the sender from the attestation
const address = await recoverMessageAddress({
message: { raw: this.p2pMessageIdentifier().to0xString() },
signature: this.signature.to0xString(),
});
// Cache the sender for later use
this.sender = EthAddress.fromString(address);
}

return this.sender;
}

toBuffer(): Buffer {
return serializeToBuffer([this.header, this.signature.length, this.signature]);
return serializeToBuffer([this.header, this.archive, this.signature]);
}

static fromBuffer(buf: Buffer | BufferReader): BlockAttestation {
const reader = BufferReader.asReader(buf);
return new BlockAttestation(reader.readObject(Header), reader.readBuffer());
return new BlockAttestation(reader.readObject(Header), reader.readObject(Fr), reader.readObject(Signature));
}

static empty(): BlockAttestation {
return new BlockAttestation(Header.empty(), Fr.ZERO, Signature.empty());
}
}
Loading
Loading