Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EVM] Update the EVM to the most recent version #3506

Closed
alexauroradev opened this issue Oct 19, 2020 · 8 comments
Closed

[EVM] Update the EVM to the most recent version #3506

alexauroradev opened this issue Oct 19, 2020 · 8 comments
Assignees
Labels
A-EVM Area: Native EVM implementation and support C-dependencies Category: Pull requests that update a dependency file C-housekeeping Category: Refactoring, cleanups, code quality

Comments

@alexauroradev
Copy link
Collaborator

The implementation of the EVM in #3257 is not the most recent. Before releasing we should update the implementation

@alexauroradev alexauroradev added A-EVM Area: Native EVM implementation and support C-housekeeping Category: Refactoring, cleanups, code quality labels Oct 19, 2020
@artob
Copy link
Contributor

artob commented Oct 21, 2020

@djsatok As discussed on the call just now, I'll take a stab at this. (Please assign the task?)

@artob
Copy link
Contributor

artob commented Oct 30, 2020

@djsatok @mikedotexe @ailisp All right, to summarize our discussion on the sync call yesterday:

We are currently pulling in Parity-Ethereum at revision openethereum/openethereum@eed630a, dated Jun 5, 2019. That roughly corresponds to upstream's 2.6.0-beta release.

Since then, Parity-Ethereum has been rebranded to OpenEthereum, and they have released the 2.7 and 3.0 branches. (They also note that due to database format changes, upgrading from 2.6.x to 2.7.x or 3.0 is a one-way only operation.)

Since the transitive dependency graph change for the full upgrade is rather large, and our code will need to be adjusted as well, we discussed on the call to do the upgrade in more than one step:

  1. Upgrading ASAP to the last 2.6 security release, 2.6.8, dated Dec 31, 2019. That already rolls us forward half a year, without (hopefully) many breaking API changes.

  2. Upgrading from 2.6 to the latest 3.0.1, dated Jun 1, 2020. That'll be painful, so we should try to schedule and phase it such that it doesn't disrupt everybody's work too badly.

Since we talked, I've upgraded my local dev env progressively through 2.6.x releases. Everything still compiles perfectly up to 2.6.2, but at 2.6.3 we do hit some API changes (never mind semantic versioning):

Screen Shot 2020-10-30 at 14 58 32

Given that, what I'd do is send a PR today for the no-brainer 2.6.2 update, and then work on making 2.6.3 (through 2.6.8) compile. Sound good, @djsatok?

@alexauroradev
Copy link
Collaborator Author

@artob, sure, no problem, let's follow suggested approach

artob added a commit that referenced this issue Oct 30, 2020
…3506

This is the last release that nearcore can pull in without errors.
Further upgrades will require code changes in near-evm-runner.
evgenykuzyakov pushed a commit that referenced this issue Oct 30, 2020
…3506 (#3542)

As a first step in #3506, upgrade OpenEthereum from [pre-2.6.0](openethereum/openethereum@eed630a) (Jun 2019) to [2.6.2](https://github.com/openethereum/openethereum/releases/tag/v2.6.2) (Aug 2019).

OpenEthereum 2.6.2 is the last release that nearcore can currently pull in without build errors. Further upgrades will require code changes in near-evm-runner. (I'm working on those.)

The upstream diff from what we currently pull in compared to 2.6.2: openethereum/openethereum@eed630a...v2.6.2
evgenykuzyakov pushed a commit that referenced this issue Nov 5, 2020
As the next step in #3506, and following up from #3542, upgrade OpenEthereum from [2.6.2](https://github.com/openethereum/openethereum/releases/tag/v2.6.2) (Aug 2019) to [2.6.8](https://github.com/openethereum/openethereum/releases/tag/v2.6.8) (Dec 2019), the last release in the 2.6.x series.

This brings us up to par on the [Istanbul](https://eth.wiki/en/roadmap/istanbul) hard fork, which incorporated many EIPs, changing gas metering and adding a new precompile as well as two new opcodes:

- [EIP-152: Add BLAKE2 compression function `F` precompile](https://eips.ethereum.org/EIPS/eip-152)
- [EIP-1108: Reduce alt_bn128 precompile gas costs](https://eips.ethereum.org/EIPS/eip-1108)
- [EIP-1344: ChainID opcode](https://eips.ethereum.org/EIPS/eip-1344) (includes a new opcode: `CHAINID`)
- [EIP-1884: Repricing for trie-size-dependent opcodes](https://eips.ethereum.org/EIPS/eip-1884) (includes a new opcode: `SELFBALANCE`)
- [EIP-2028: Transaction data gas cost reduction](https://eips.ethereum.org/EIPS/eip-2028)
- [EIP-2200: Structured Definitions for Net Gas Metering](https://eips.ethereum.org/EIPS/eip-2200)

This PR includes an implementation of the `CHAINID` opcode. (The `SELFBALANCE` opcode doesn't need anything from our runtime.)

The upstream changelog for 2.6.8 is at: https://github.com/openethereum/openethereum/blob/v2.6.8/CHANGELOG.md

The upstream diff from 2.6.2 to 2.6.8 is at: openethereum/openethereum@v2.6.2...v2.6.8
@artob
Copy link
Contributor

artob commented Nov 6, 2020

Consider run runtime/near-evm-runner/tests/standard_ops.rs test and make it pass again locally. And add a test case for blake2 precompile function

Originally posted by @ailisp in #3558 (comment)

artob added a commit that referenced this issue Nov 6, 2020
Note: this is a work in progress, it doesn't build as yet.
@artob
Copy link
Contributor

artob commented Nov 6, 2020

To summarize the current state of things:

  1. As a first step, we upgraded to OpenEthereum 2.6.2 in Upgrade to OpenEthereum 2.6.2 #3542. That rolled us forward about 3 months (from Jun 2019 to Aug 2019).

  2. Thereafter, we upgraded to OpenEthereum 2.6.8 in Upgrade to OpenEthereum 2.6.8 (Istanbul support) #3558. That rolled us forward another 4 months (from Aug 2019 to Dec 2019), and brought us to parity with the Istanbul hard fork. The upgrade chiefly required implementing a new API that is called when the new CHAINID opcode is executed.

  3. I'm currently working on the upgrade to OpenEthereum 2.7.2. That will roll us forward another 2 months (from Dec 2019 to Feb 2020). And that will get us to a stable release: the whole 2.6.x series is considered beta, with 2.7.x being the stable branch. To complete the upgrade, there are a number of dependency issues and build errors to address:

Screen Shot 2020-11-06 at 16 11 54

Screen Shot 2020-11-06 at 16 12 21

  1. As a final step, we are planning to upgrade to OpenEthereum 3.0.1, the current stable release at the time of writing. That will roll us forward another 4 months (from Feb 2020 to Jun 2020), for a total of 12 months forward from the starting point.

@artob
Copy link
Contributor

artob commented Nov 6, 2020

Related to this issue, here’s a Solidity/OpenEthereum version cross reference that I compiled at @djsatok's request:

https://gist.github.com/artob/c2b5dc6c834134f3e7c1ef86232b416d

artob added a commit that referenced this issue Nov 10, 2020
Note: this is a work in progress, it doesn't build as yet.
artob added a commit that referenced this issue Nov 12, 2020
Note: this is a work in progress, it doesn't build as yet.
artob added a commit that referenced this issue Nov 12, 2020
Note: this is a work in progress, it doesn't build as yet.
evgenykuzyakov pushed a commit that referenced this issue Nov 12, 2020
As the next step in #3506, and following up from #3542 and #3558, upgrade Parity Ethereum (now OpenEthereum) from [2.6.8] (Dec 2019) to [2.7.2] (Feb 2020), the last release in the 2.x series.

This harmonizes our dependency graph with upstream, fixing numerous build errors from the upgrade and eliminating multiple versions of the same crates. This concerns the [bn], [ethabi], [ethabi-contract], [ethabi-derive], [ethereum-types], [keccak-hash], [primitive-types], and [uint] crates as well as their transitive dependencies and their interdependencies.

The [near/bn] fork exists to upgrade [bn]'s [rand] dependency. That's necessary because of a particularly complex dependency issue (rust-random/rand#645) in old versions of rand. We will eliminate the fork going forward once upstream upgrades their dependency (I will open a pull request for them).

The upstream changelog for 2.7.2 is at: https://github.com/openethereum/openethereum/blob/v2.7.2/CHANGELOG.md

The upstream diff from 2.6.8 to 2.7.2 is at: openethereum/openethereum@v2.6.8...v2.7.2

[2.6.8]: https://github.com/openethereum/openethereum/releases/tag/v2.6.8
[2.7.2]: https://github.com/openethereum/openethereum/releases/tag/v2.7.2

[near/bn]: https://github.com/near/bn
[bn]: https://crates.io/crates/bn
[ethabi]: https://crates.io/crates/ethabi
[ethabi-contract]: https://crates.io/crates/ethabi-contract
[ethabi-derive]: https://crates.io/crates/ethabi-derive
[ethereum-types]: https://crates.io/crates/ethereum-types
[keccak-hash]: https://crates.io/crates/keccak-hash
[primitive-types]: https://crates.io/crates/primitive-types
[rand]: https://crates.io/crates/rand
[uint]: https://crates.io/crates/uint
@artob
Copy link
Contributor

artob commented Nov 14, 2020

As discussed on the weekly call today, the current state of this task is that we've now completed and merged the 2.7.2 upgrade (#3595), and it makes sense to freeze the dependency at that release for now.

OpenEthereum surprised us a week ago by releasing 3.1.0 which they had been developing in another repository. OpenEthereum 3.1.0 isn't a continuation of the 3.0.x lineage, but rather a fresh fork of the old 2.5.x branch. The 3.0.x branch appears to be considered a dead end, now.

In investigating the new branch this week, my assessment, and the team's consensus, is that upgrading to OpenEthereum 3.1.0 is too risky right now, so we will build on 2.7.2 for the time being.

The 2.7.2 version was the last Parity Ethereum release, before the OpenEthereum team took over. So, it is relatively popular and well tested. It will serve us as a “good enough” release until the Berlin hard fork takes effect, which may require an upgrade to 3.1.x, or else some backporting work to 2.7.x.

@djsatok @evgenykuzyakov I think we can close this task, for now, as I don't think there's anything further actionable here until it's time to prep for the Berlin hard fork (in January, perhaps, if Berlin doesn't get delayed further).

@artob artob assigned alexauroradev and unassigned artob Nov 14, 2020
@artob artob added the C-dependencies Category: Pull requests that update a dependency file label Nov 14, 2020
@alexauroradev
Copy link
Collaborator Author

Closed by #3595 and previous PRs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-EVM Area: Native EVM implementation and support C-dependencies Category: Pull requests that update a dependency file C-housekeeping Category: Refactoring, cleanups, code quality
Projects
None yet
Development

No branches or pull requests

2 participants