Fluffy: Portal EVM call #3119

bhartnett · 2025-03-04T11:13:58Z

This Portal EVM implementation uses the Nimbus in-memory EVM to execute transactions using the portal state network state data. Currently only call is supported.

Rather than wire in the portal state lookups into the EVM directly, the approach taken here is to optimistically execute the transaction multiple times with the goal of building the correct access list so that we can then lookup the required state from the portal network, store the state in the in-memory EVM and then finally execute the transaction using the correct state. The Portal EVM makes use of data in memory during the call and therefore each piece of state is never
fetched more than once. We know we have found the correct access list if it doesn't change after another execution of the transaction.

The assumption here is that network lookups for state data are generally much slower than the time it takes to execute a transaction in the EVM and therefore executing the transaction multiple times should not significantly slow down the
call given that we gain the ability to fetch the state concurrently.

There are multiple reasons for choosing this approach:

Firstly updating the existing Nimbus EVM to support using a different state backend (portal state in this case) is difficult and would require making non-trivial changes to the EVM.
This new approach allows us to look up the state concurrently in the event that multiple new state keys are discovered after executing the transaction. This should in theory result in improved performance for certain scenarios. The default approach where the state lookups are wired directly into the EVM gives the worst case performance because all state accesses inside the EVM are completely sequential.

bhartnett · 2025-03-04T11:30:42Z

execution_chain/common/common.nim

@@ -217,7 +218,8 @@ proc init(com         : CommonRef,
  # By default, history begins at genesis.
  com.startOfHistory = GENESIS_PARENT_HASH

-  com.initializeDb()
+  if initializeDb:
+    com.initializeDb()


Made this configurable so that it can be disabled in Fluffy which doesn't need it. Removes the logs also which are not relevant in the portal context.

bhartnett · 2025-03-04T11:32:39Z

execution_chain/stateless/multi_keys.nim

+    if not equalsStorageMode(kd1.storageKeys, kd2.storageKeys):
+      return false
+
+  return true


This code is required to check if the keys have changed. Is there a better way to compare case object ref types?

bhartnett · 2025-03-04T11:35:43Z

fluffy/evm/portal_evm.nim

+
+      # Store fetched state in the in-memory EVM
+      for q in accountQueries:
+        let acc = (await q.accFut).valueOr:


When awaiting these futures here the internal future type raises a CatchableError when the async proc called actually raises a CancelledError. Not sure what is causing this and how I should handle it.

bhartnett · 2025-03-04T11:37:09Z

fluffy/portal_node.nim

+      if historyNetwork.isSome() and stateNetwork.isSome():
+        Opt.some(PortalEvm.init(historyNetwork.get(), stateNetwork.get()))
+      else:
+        Opt.none(PortalEvm),


The PortalEvm is created only when both the history network and the state network are running. Should we start the portal evm using an alternative method such as having a separate cli argument?

Conceptually I do see the PortalEvm as an optional part that is not required to run a pure portal node. That is a node that only does its duties on a Portal Network protocol level and does not provide higher level API such as the EL JSON API. (We could even rewrite the types to have PortalClient or PortalLightClient which has the subtypes of PortalNode and PortalEvm.)

In that sense it should be possible to run a portal node with history/state/beacon subnetworks enabled but the evm not running. This could be achieved with an extra cli argument.

Another way to achieve this (with perhaps better UX?) could be to enable it based on enabling of linked functionality. I believe in this case that would just mean in case the EL (eth) JSON-RPC API gets enabled, which it currently default does.

This would avoid having to add additional "Not supported" errors for certain JSON-RPC endpoints in case you would use a separate extra cli argument.

If we would go that route, then I am uncertain whether we should then keep eth as default enabled, but I'm leaning towards yes.

Conceptually I do see the PortalEvm as an optional part that is not required to run a pure portal node. That is a node that only does its duties on a Portal Network protocol level and does not provide higher level API such as the EL JSON API. (We could even rewrite the types to have PortalClient or PortalLightClient which has the subtypes of PortalNode and PortalEvm.)

In that sense it should be possible to run a portal node with history/state/beacon subnetworks enabled but the evm not running. This could be achieved with an extra cli argument.

Another way to achieve this (with perhaps better UX?) could be to enable it based on enabling of linked functionality. I believe in this case that would just mean in case the EL (eth) JSON-RPC API gets enabled, which it currently default does.

This would avoid having to add additional "Not supported" errors for certain JSON-RPC endpoints in case you would use a separate extra cli argument.

If we would go that route, then I am uncertain whether we should then keep eth as default enabled, but I'm leaning towards yes.

Sure, I'll remove it from PortalNode and only create the PortalEvm instance when eth rpc is enabled. I think this is the simplest way for now considering that it will only be used in the eth rpc at this point. In the future when we implement the transaction gossip sub-network the requirements may change and at that point it could be moved somewhere else if/when needed.

bhartnett · 2025-03-04T11:38:44Z

fluffy/rpc/rpc_eth_api.nim

+  #   maxFeePerGas*: Opt[Quantity]         # (optional) MaxFeePerGas is the maximum fee per gas offered, in wei.
+  #   maxPriorityFeePerGas*: Opt[Quantity] # (optional) MaxPriorityFeePerGas is the maximum miner tip per gas offered, in wei.
+  #   value*: Opt[UInt256]     # (optional) Integer of the value sent with this transaction.
+  #   nonce*: Opt[Quantity]    # (optional) integer of a nonce. This allows to overwrite your own pending transactions that use the same nonce


The eth_call implementation here is not the core focus of this PR. It is simply used for some basic manual testing. I will create another future PR which refines and further tests the various eth_call scenarios.

bhartnett · 2025-03-04T11:39:40Z

fluffy/network/state/state_network.nim

@@ -262,10 +262,10 @@ proc processContentLoop(n: StateNetwork) {.async: (raises: []).} =
 proc statusLogLoop(n: StateNetwork) {.async: (raises: []).} =
  try:
    while true:
+      await sleepAsync(60.seconds)


Moved the sleep so that during startup the logs look better/cleaner.

…sting call.

bhartnett added 10 commits February 26, 2025 13:49

Start using Nimbus EVM.

02d8569

Implement proof of concept implementation using Nimbus EVM.

43472d5

Cleanup.

1164a11

Make CommonRef db initialization configurable.

86d0d82

Return error string. Use txFrame dispose.

4d2f21b

Make state lookups concurrent.

0cc05d8

Move equals into multi_keys.nim.

602c815

Merge branch 'master' into fluffy-evm

497303d

Add some documentation and comments. Tests for multikeys equals.

64a69bb

Add logging.

3102ebe

bhartnett marked this pull request as draft March 4, 2025 11:14

bhartnett requested a review from kdeme March 4, 2025 11:27

bhartnett commented Mar 4, 2025

View reviewed changes

bhartnett added 6 commits March 4, 2025 23:05

Disable linking to RocksDb in Fluffy.

55fb634

Merge branch 'master' into fluffy-evm

63a1268

Fix issue discovered when calling another contract from within an exi…

76e0ed2

…sting call.

Fix copyright.

d3b27b2

Add to address to fetched code.

52ed102

Merge branch 'master' into fluffy-evm

9496662

bhartnett mentioned this pull request Mar 7, 2025

Fluffy and the EVM #1162

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fluffy: Portal EVM call #3119

Fluffy: Portal EVM call #3119

bhartnett commented Mar 4, 2025 •

edited

Loading

bhartnett Mar 4, 2025

bhartnett Mar 4, 2025

bhartnett Mar 4, 2025

bhartnett Mar 4, 2025

kdeme Mar 7, 2025

bhartnett Mar 10, 2025

bhartnett Mar 4, 2025

bhartnett Mar 4, 2025

Fluffy: Portal EVM call #3119

Are you sure you want to change the base?

Fluffy: Portal EVM call #3119

Conversation

bhartnett commented Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhartnett commented Mar 4, 2025 •

edited

Loading