Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7935: Add E2StoreReader #8329

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions util/src/main/java/org/hyperledger/besu/util/e2/E2BeaconState.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

/**
* Represents a beacon state in an E2 file
*
* @param beaconState The beacon state
* @param slot The slot number
*/
public record E2BeaconState(byte[] beaconState, int slot) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need the beacon types as we won't be writing or reading the beacon data from anywhere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was initially writing a generic e2 file reader. Are we unlikely to use era files for populating the chain beyond the merge? I realise we can populate it through the p2p network, or potentially from portal, but it shouldn't be difficult to support this too.

25 changes: 25 additions & 0 deletions util/src/main/java/org/hyperledger/besu/util/e2/E2BlockIndex.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

import java.util.List;

/**
* Represents a block index in an E2 file
*
* @param startingSlot The first slot number indexed by this block index
* @param indexes The indexes of the blocks indexed by this block index
*/
public record E2BlockIndex(long startingSlot, List<Long> indexes) {}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

/**
* Represents an execution block body in an E2 file
*
* @param block The execution block
* @param slot The slot number
*/
public record E2ExecutionBlockBody(byte[] block, int slot) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slot should be blockIndex from my understanding of the spec

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

/**
* Represents an execution block header in an E2 file
*
* @param header The execution block header
* @param slot The slot number
*/
public record E2ExecutionBlockHeader(byte[] header, int slot) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slot should be blockIndex

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

/**
* Represents an execution block's transaction receipts in an E2 file
*
* @param receipts The execution block's transaction receipts
* @param slot The slot number
*/
public record E2ExecutionBlockReceipts(byte[] receipts, int slot) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slot should be blockIndex

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

/**
* Represents a signed beacon block in an E2 file
*
* @param signedBeaconBlock The signed beacon block
* @param slot The slot number
*/
public record E2SignedBeaconBlock(byte[] signedBeaconBlock, int slot) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, we don't need this type

25 changes: 25 additions & 0 deletions util/src/main/java/org/hyperledger/besu/util/e2/E2SlotIndex.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

import java.util.List;

/**
* Represents a slot index in an E2 file
*
* @param startingSlot The first slot number indexed by this slot index
* @param indexes The indexes of the slots indexed by this slot index
*/
public record E2SlotIndex(long startingSlot, List<Long> indexes) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also don't need this

Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

/** A Listener interface for listening to an E2StoreReader */
public interface E2StoreReaderListener {
/**
* Handles the supplied E2BeaconState
*
* @param beaconState the E2BeaconState
*/
void handleBeaconState(E2BeaconState beaconState);

/**
* Handles the supplied E2SlotIndex
*
* @param slotIndex the E2SlotIndex
*/
void handleSlotIndex(E2SlotIndex slotIndex);

/**
* Handles the supplied E2SignedBeaconBlock
*
* @param signedBeaconBlock the E2SignedBeaconBlock
*/
void handleSignedBeaconBlock(E2SignedBeaconBlock signedBeaconBlock);
Comment on lines +24 to +38
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can remove the beacon handlers, they won't be in the era1 files we use


/**
* Handles the supplied E2ExecutionBlockHeader
*
* @param executionBlockHeader the E2ExecutionBlockHeader
*/
void handleExecutionBlockHeader(E2ExecutionBlockHeader executionBlockHeader);

/**
* Handles the supplied E2ExecutionBlockBody
*
* @param executionBlockBody the E2ExecutionBlockBody
*/
void handleExecutionBlockBody(E2ExecutionBlockBody executionBlockBody);

/**
* Handles the supplied E2ExecutionBlockReceipts
*
* @param executionBlockReceipts the E2ExecutionBlockReceipts
*/
void handleExecutionBlockReceipts(E2ExecutionBlockReceipts executionBlockReceipts);

/**
* Handles the supplied E2BlockIndex
*
* @param blockIndex the E2BlockIndex
*/
void handleBlockIndex(E2BlockIndex blockIndex);
}
83 changes: 83 additions & 0 deletions util/src/main/java/org/hyperledger/besu/util/e2/E2Type.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.util.e2;

import java.util.HexFormat;

/** An enumeration of known sections of e2 store files */
public enum E2Type {
/** An empty section */
EMPTY(new byte[] {0x00, 0x00}),
/** A snappy compressed, SSZ encoded, signed beacon block */
COMPRESSED_SIGNED_BEACON_BLOCK(new byte[] {0x01, 0x00}),
/** A snappy compressed, SSZ encoded, beacon state */
COMPRESSED_BEACON_STATE(new byte[] {0x02, 0x00}),
Comment on lines +24 to +26
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also can remove these as we don't need to parse era files

/** A snappy compressed execution block header */
COMPRESSED_EXECUTION_BLOCK_HEADER(new byte[] {0x03, 0x00}),
/** A snappy compressed execution block body */
COMPRESSED_EXECUTION_BLOCK_BODY(new byte[] {0x04, 0x00}),
/** A snappy compressed list of execution block transaction receipts */
COMPRESSED_EXECUTION_BLOCK_RECEIPTS(new byte[] {0x05, 0x00}),
/** The total difficulty */
TOTAL_DIFFICULTY(new byte[] {0x06, 0x00}),
/** The accumulator */
ACCUMULATOR(new byte[] {0x07, 0x00}),
/** A version section */
VERSION(new byte[] {0x65, 0x32}),
/** An execution block index */
BLOCK_INDEX(new byte[] {0x66, 0x32}),
/** A slot index */
SLOT_INDEX(new byte[] {0x69, 0x32}),
;
private final byte[] typeCode;

E2Type(final byte[] typeCode) {
this.typeCode = typeCode;
}

/**
* Gets the type code
*
* @return the type code
*/
public byte[] getTypeCode() {
return typeCode;
}

/**
* Gets the E2Type corresponding to the supplied typeCode
*
* @param typeCode the typeCode to find the corresponding E2Type for
* @return the E2Type corresponding to the supplied typeCode
* @throws IllegalArgumentException if there is no E2Type corresponding to the supplied typeCode
*/
public static E2Type getForTypeCode(final byte[] typeCode) {
if (typeCode == null || typeCode.length != 2) {
throw new IllegalArgumentException("typeCode must be 2 bytes");
}

E2Type result = null;
for (E2Type e2Type : values()) {
if (e2Type.typeCode[0] == typeCode[0] && e2Type.typeCode[1] == typeCode[1]) {
result = e2Type;
}
}
if (result == null) {
throw new IllegalArgumentException(
"typeCode " + HexFormat.of().formatHex(typeCode) + " is not recognised");
}
return result;
}
}
Loading