Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: Add coverage results in README #66

Merged
merged 3 commits into from
Jun 18, 2024
Merged

doc: Add coverage results in README #66

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b7c851d
doc: Add coverage results in README
Nashtare Jun 15, 2024
e07ac0a
Add space
Nashtare Jun 15, 2024
70d6187
Apply suggestion
Nashtare Jun 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,91 @@ would not be provable anyway. For tests that have an acceptable transaction `gas
alter the latter to be `0xFFFFFFFF` (i.e. the maximum value fitting in a `u32`). If the runner manages to generate a valid witness / proof
for this altered test, we log it as valid. If it fails, then we flag the test as ignored.

## Coverage [zk_evm v0.4.0]

The results below have been obtained against [zk_evm v0.4.0](https://github.com/0xPolygonZero/zk_evm/releases/tag/v0.4.0).

All test vectors have been fetched from [ethereum/legacytests](https://github.com/ethereum/legacytests/tree/master) from
commit [8077d24](https://github.com/ethereum/legacytests/commit/8077d241740de5a448e41156fd33740f562d3b56).

The total number of tests below excludes:

- tests with invalid block RLP encoding (from which all the test data is retrieved)
- tests with a txn gas used greater than $2^{32}-1$, see [Note on ignored tests](#note-on-ignored-tests).
- stress tests with a zkEVM CPU overhead greater than Goldilocks' two-adic subgroup size.


| Total | :white_check_mark: | :x: | Coverage |
|:-----:|:------------------:|:------:|:----------:|
| 14904 | 14896 | 8 [^1] | 99.95% |

[^1]: These tests have an impossible initial configuration (i.e. empty accounts with non-empty storage),
and end up not being provable.

### Extended results

| Test Folder Name | Total | :white_check_mark: | :x: | Cov |
|:------------------------------------:|:-----:|:------------------:|:---:|:------:|
| Shanghai | 26 | 26 | 0 | 100.00 |
| stArgsZeroOneBalance | 96 | 96 | 0 | 100.00 |
| stAttackTest | 2 | 2 | 0 | 100.00 |
| stBadOpcode | 340 | 340 | 0 | 100.00 |
| stBugs | 9 | 9 | 0 | 100.00 |
| stCallCodes | 86 | 86 | 0 | 100.00 |
| stCallCreateCallCodeTest | 55 | 55 | 0 | 100.00 |
| stCallDelegateCodesCallCodeHomestead | 58 | 58 | 0 | 100.00 |
| stCallDelegateCodesHomestead | 58 | 58 | 0 | 100.00 |
| stChainId | 2 | 2 | 0 | 100.00 |
| stCodeCopyTest | 2 | 2 | 0 | 100.00 |
| stCodeSizeLimit | 7 | 7 | 0 | 100.00 |
| stCreate2 | 184 | 181 | 3 | 98.37 |
| stCreateTest | 204 | 204 | 0 | 100.00 |
| stDelegatecallTestHomestead | 31 | 31 | 0 | 100.00 |
| stEIP150Specific | 25 | 25 | 0 | 100.00 |
| stEIP150singleCodeGasPrices | 340 | 340 | 0 | 100.00 |
| stEIP1559 | 949 | 949 | 0 | 100.00 |
| stEIP158Specific | 8 | 8 | 0 | 100.00 |
| stEIP2930 | 140 | 140 | 0 | 100.00 |
| stEIP3607 | 5 | 5 | 0 | 100.00 |
| stExample | 38 | 38 | 0 | 100.00 |
| stExtCodeHash | 69 | 68 | 1 | 98.56 |
| stHomesteadSpecific | 5 | 5 | 0 | 100.00 |
| stInitCodeTest | 22 | 22 | 0 | 100.00 |
| stLogTests | 46 | 46 | 0 | 100.00 |
| stMemExpandingEIP150Calls | 10 | 10 | 0 | 100.00 |
| stMemoryStressTest | 82 | 79 | 0 | 100.00 |
| stMemoryTest | 567 | 567 | 0 | 100.00 |
| stNonZeroCallsTest | 24 | 24 | 0 | 100.00 |
| stPreCompiledContracts | 956 | 956 | 0 | 100.00 |
| stPreCompiledContracts2 | 246 | 246 | 0 | 100.00 |
| stQuadraticComplexityTest | 28 | 28 | 0 | 100.00 |
| stRandom | 310 | 310 | 0 | 100.00 |
| stRandom2 | 221 | 221 | 0 | 100.00 |
| stRecursiveCreate | 2 | 2 | 0 | 100.00 |
| stRefundTest | 26 | 26 | 0 | 100.00 |
| stReturnDataTest | 247 | 247 | 0 | 100.00 |
| stRevertTest | 270 | 270 | 0 | 100.00 |
| stSLoadTest | 1 | 1 | 0 | 100.00 |
| stSStoreTest | 475 | 471 | 4 | 99.16 |
| stSelfBalance | 42 | 42 | 0 | 100.00 |
| stShift | 42 | 42 | 0 | 100.00 |
| stSolidityTest | 23 | 23 | 0 | 100.00 |
| stSpecialTest | 19 | 19 | 0 | 100.00 |
| stStackTests | 375 | 375 | 0 | 100.00 |
| stStaticCall | 455 | 455 | 0 | 100.00 |
| stStaticFlagEnabled | 34 | 34 | 0 | 100.00 |
| stSystemOperationsTest | 83 | 83 | 0 | 100.00 |
| stTimeConsuming | 5187 | 5187 | 0 | 100.00 |
| stTransactionTest | 162 | 162 | 0 | 100.00 |
| stTransitionTest | 6 | 6 | 0 | 100.00 |
| stWalletTest | 46 | 46 | 0 | 100.00 |
| stZeroCallsRevert | 16 | 16 | 0 | 100.00 |
| stZeroCallsTest | 24 | 24 | 0 | 100.00 |
| stZeroKnowledge2 | 519 | 519 | 0 | 100.00 |
| stZeroKnowledge | 944 | 944 | 0 | 100.00 |
| VMTests | 649 | 649 | 0 | 100.00 |


## Other

[Polygon Hermez](https://github.com/0xPolygonHermez) is doing something similar [here](https://github.com/0xPolygonHermez/zkevm-testvectors).
Expand Down
Loading