Mock implementation of all `RPC` methods under the `eth` namespace #10

m-Peter · 2023-11-01T13:39:44Z

Work towards:

Description

Creates an RPC server, registers a service for the eth namespace, and serves JSON-RPC on a TCP listener.

The BlockChainAPI service which is registered on the eth namespace, implements all the RPC methods described in the above issues. Their implementations either return empty / mock data, or a not implemented/support error according to the description of each issue.

Running this locally, results in:

➜  flow-evm-grpc git:(go-ethereum-json-rpc-example) go run .                                    
Request:  {"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params": []}
Response:  {"jsonrpc":"2.0","id":1,"result":"0x3ecc3d0"}

Request:  {"jsonrpc":"2.0","id":2,"method":"eth_chainId","params": []}
Response:  {"jsonrpc":"2.0","id":2,"result":"0x309"}

Request:  {"jsonrpc":"2.0","id":3,"method":"eth_syncing","params": []}
Response:  {"jsonrpc":"2.0","id":3,"result":false}

Request:  {"jsonrpc":"2.0","id":4,"method":"eth_getBalance","params": ["0x407d73d8a49eeb85d32cf465507dd71d507100c1"]}
Response:  {"jsonrpc":"2.0","id":4,"result":"0x65"}

Request:  {"jsonrpc":"2.0","id":5,"method":"eth_getBlockTransactionCountByNumber","params": ["0x4E4ee"]}
Response:  {"jsonrpc":"2.0","id":5,"result":"0x20a"}

For contributor use:

Targeted PR against master branch
Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
Code follows the standards mentioned here.
Updated relevant documentation
Re-reviewed Files changed in the Github PR explorer
Added appropriate labels

…m and wiring with flow-go-sdk

devbugging · 2023-11-08T19:36:55Z

Sorry for late reply, let me take a look.

devbugging · 2023-11-09T12:41:51Z

@m-Peter I would suggest a different approach. I would first focus on defining the schema for the data you would get from the Flow network related to EVM blocks and transaction results. I would create a mock version and then focus on the indexer of that data. Then I would generate endpoitns based on the ethereum rpc spec. Abstracting away the implementation details of how you get the data from Flow network about the blocks and transactions will allow you to adapt it as the project evolves. The schema might stay more or less similar. Having mocks in place for getting data you can also then implement the proofs you need.

ramtinms · 2023-11-09T18:40:11Z

Even though its a draft, still looks good to me, would review again when it becomes something ready to be merged

peterargue

Looking over the issues, all of the endpoints are accounted for here.

I think it's probably worthwhile to start structuring the project a bit. Rather than putting everything within a main.go, how about a structure like:

/
├─ api/
│  ├─ api.go
│  ├─ server.go
│  ├─ models.go
├─ cmd/
│  ├─ server/
│  │  ├─ main.go

I'm definitely not attached to any specific structure, and this can be done in a separate PR.

With that said, I think this is a fine foundation if you want to merge it in and start iterating in more PRs

devbugging · 2023-11-20T12:17:45Z

main.go

+	ChainID    *hexutil.Big      `json:"chainId,omitempty"`
+}
+
+type BlockChainAPI struct {


I would say calling this grpcServer would be more appropriate, because in the future there might be multiple implementation of the "ethereum API".

It would also be useful to define an interface for this API so you can mock out for testing (maybe too early for this tho)

I would say calling this grpcServer would be more appropriate, because in the future there might be multiple implementation of the "ethereum API".

This is not the actual JSON-RPC server, but an object that has to adhere to the Ethereum Specification. E.g.

server := rpc.NewServer() server.RegisterName("eth", &BlockChainAPI{})

Each exported method of this object (for example BlockNumber) is wired to an RPC method -> eth_blockNumber

It would also be useful to define an interface for this API so you can mock out for testing (maybe too early for this tho)

That's indeed a good idea 👍 And I will also break it down to more types (TransactionsAPI, AccountsAPI, FiltersAPI).

main.go

devbugging · 2023-11-20T12:20:35Z

main.go

+// returned, regardless of the current head block. We used to return an error when the chain
+// wasn't synced up to a block where EIP-155 is enabled, but this behavior caused issues
+// in CL clients.
+func (api *BlockChainAPI) ChainId() *hexutil.Big {


Is there maybe a protobuff you can use to generate the methods needed for eth rpc API?

Unfortunately there is no such protobuf in the go-ethereum repository. Most like these methods are hand-written.

devbugging · 2023-11-20T12:21:00Z

main.go

+// accessListResult returns an optional accesslist
+// It's the result of the `debug_createAccessList` RPC call.
+// It contains an error if the transaction itself failed.
+type accessListResult struct {


I would put this and some other types under a model package.

Good point 👍 .
I have added a basic project structure, as pointed out by Peter 62a9b84.
For now I have added them in a models.go file, under the api package.

devbugging · 2023-11-20T12:22:27Z

main.go

+}
+
+// eth_feeHistory (transaction fee history)
+// FeeHistory returns the fee market history.


Explain all the arguments

Added in ed4fe06
I will go over all the methods and get the argument explanation from https://ethereum.github.io/execution-apis/api-documentation/

devbugging · 2023-11-20T12:23:09Z

main.go

+func (s *BlockChainAPI) CreateAccessList(
+	ctx context.Context,
+	args TransactionArgs,
+	blockNrOrHash *rpc.BlockNumberOrHash,


nit:

Suggested change

blockNrOrHash *rpc.BlockNumberOrHash,

BlockNumberOrHash *rpc.BlockNumberOrHash,

Updated in 62a9b84

devbugging · 2023-11-20T12:26:24Z

main.go

+func (s *BlockChainAPI) GetStorageAt(
+	ctx context.Context,
+	address common.Address,
+	hexKey string, blockNrOrHash rpc.BlockNumberOrHash,


What is hexKey, maybe publicKey? also might be better to not put hex in the name, since you will validate the input anyway and you can detect the encoding. You should put that in the comment.

Good point 👍 In the specification it is written:
Returns the value from a storage position at a given address. So hexKey is actually the storage position.
And the accepted value is a hex encoded 256 bit unsigned integer. Most of the naming comes from the go-ethereum` repository, but I will rename the arguments accordingly, to make them more readable.

Updated in 71ef6d1

devbugging · 2023-11-20T12:28:19Z

main.go

+//     only the transaction hash is returned.
+func (s *BlockChainAPI) GetBlockByNumber(
+	ctx context.Context,
+	number rpc.BlockNumber,


I would define enums for above casses.

Actually the go-ethereum repository already defines the following constants:

type BlockNumber int64 const ( SafeBlockNumber = BlockNumber(-4) FinalizedBlockNumber = BlockNumber(-3) LatestBlockNumber = BlockNumber(-2) PendingBlockNumber = BlockNumber(-1) EarliestBlockNumber = BlockNumber(0) )

So we can use these.

devbugging · 2023-11-20T12:28:52Z

main.go

+	number rpc.BlockNumber,
+	fullTx bool,
+) (map[string]interface{}, error) {
+	return map[string]interface{}{}, nil


Why returning map[string]any? wouldn't it be better to define a type for it

I am assuming that's because the method has to return nil. At least that's what I read on the specification page, and the actual implementation:

block, err := s.b.BlockByNumber(ctx, number) if block != nil && err == nil { response, err := s.rpcMarshalBlock(ctx, block, true, fullTx) if err == nil && number == rpc.PendingBlockNumber { // Pending blocks need to nil out a few fields for _, field := range []string{"hash", "nonce", "miner"} { response[field] = nil } } return response, err } return nil, err

And there's a special handling for pending blocks, where they have to set 3 fields to nil value.

devbugging · 2023-11-20T12:32:31Z

I left some comments, I agree with @peterargue some of the structure you should already put in place, you don't have to overdo it at this early state, but also having everything in main will make more changes at later point to refactor. With the same thinking be careful about exposing implementation details in names etc (left some comments on that).

m-Peter · 2023-11-21T13:00:44Z

@sideninja I have addressed / answered all of the comments. Could you take another look? 🙏

Example of utilising the JSON-RPC server implementation of go-ethereu…

16bdaac

…m and wiring with flow-go-sdk

m-Peter self-assigned this Nov 1, 2023

m-Peter marked this pull request as draft November 1, 2023 13:40

franklywatson requested review from ramtinms and peterargue November 6, 2023 19:49

ramtinms approved these changes Nov 9, 2023

View reviewed changes

m-Peter mentioned this pull request Nov 16, 2023

Flow EVM: JSON-RPC gateway phase #1 grant proposal onflow/developer-grants#237

Closed

3 tasks

Implement all the RPC methods under the 'eth' namespace

0499b22

m-Peter changed the title ~~Example of utilising the JSON-RPC server implementation of go-ethereum and wiring with flow-go-sdk~~ Mock implementation of all the RPC methods under the eth namespace Nov 17, 2023

m-Peter marked this pull request as ready for review November 17, 2023 13:29

m-Peter changed the title ~~Mock implementation of all the RPC methods under the eth namespace~~ Mock implementation of all RPC methods under the eth namespace Nov 17, 2023

m-Peter requested a review from ramtinms November 17, 2023 13:51

peterargue approved these changes Nov 18, 2023

View reviewed changes

devbugging reviewed Nov 20, 2023

View reviewed changes

main.go Outdated Show resolved Hide resolved

devbugging reviewed Nov 20, 2023

View reviewed changes

Add a basic project structure

62a9b84

m-Peter force-pushed the go-ethereum-json-rpc-example branch from e1cc0ea to 62a9b84 Compare November 20, 2023 16:57

m-Peter added 2 commits November 20, 2023 19:53

Change module name to flow-evm-gateway to match repository name

f8a4f57

Add documentation for arguments in FeeHistory method

ed4fe06

Rename hexKey argument in GetStorageAt method

71ef6d1

m-Peter requested a review from devbugging November 21, 2023 13:00

devbugging approved these changes Nov 21, 2023

View reviewed changes

m-Peter merged commit 7156cee into onflow:main Nov 21, 2023

m-Peter deleted the go-ethereum-json-rpc-example branch November 21, 2023 19:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mock implementation of all `RPC` methods under the `eth` namespace #10

Mock implementation of all `RPC` methods under the `eth` namespace #10

m-Peter commented Nov 1, 2023 •

edited

Loading

devbugging commented Nov 8, 2023

devbugging commented Nov 9, 2023 •

edited

Loading

ramtinms commented Nov 9, 2023

peterargue left a comment

devbugging Nov 20, 2023

devbugging Nov 20, 2023

m-Peter Nov 21, 2023

devbugging Nov 20, 2023

m-Peter Nov 21, 2023

devbugging Nov 20, 2023

m-Peter Nov 21, 2023

devbugging Nov 20, 2023

m-Peter Nov 21, 2023

devbugging Nov 20, 2023

m-Peter Nov 21, 2023

devbugging Nov 20, 2023

m-Peter Nov 20, 2023

m-Peter Nov 21, 2023

devbugging Nov 20, 2023

m-Peter Nov 21, 2023

devbugging Nov 20, 2023

m-Peter Nov 21, 2023

devbugging commented Nov 20, 2023

m-Peter commented Nov 21, 2023

	blockNrOrHash *rpc.BlockNumberOrHash,
	BlockNumberOrHash *rpc.BlockNumberOrHash,

Mock implementation of all RPC methods under the eth namespace #10

Mock implementation of all RPC methods under the eth namespace #10

Conversation

m-Peter commented Nov 1, 2023 • edited Loading

Description

devbugging commented Nov 8, 2023

devbugging commented Nov 9, 2023 • edited Loading

ramtinms commented Nov 9, 2023

peterargue left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devbugging commented Nov 20, 2023

m-Peter commented Nov 21, 2023

Mock implementation of all `RPC` methods under the `eth` namespace #10

Mock implementation of all `RPC` methods under the `eth` namespace #10

m-Peter commented Nov 1, 2023 •

edited

Loading

devbugging commented Nov 9, 2023 •

edited

Loading