Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

recent anvil --state/dump-state changes are broken #7502

Closed
2 tasks done
samlaf opened this issue Mar 26, 2024 · 10 comments · Fixed by #8160
Closed
2 tasks done

recent anvil --state/dump-state changes are broken #7502

samlaf opened this issue Mar 26, 2024 · 10 comments · Fixed by #8160
Labels
T-bug Type: bug

Comments

@samlaf
Copy link
Contributor

samlaf commented Mar 26, 2024

Component

Anvil

Have you ensured that all of these are up to date?

  • Foundry
  • Foundryup

What version of Foundry are you on?

forge 0.2.0 (563e062 2024-03-26T00:16:13.950236000Z)

What command(s) is the bug in?

anvil --state

Operating System

macOS (Apple Silicon)

Describe the bug

There was an upgrade to anvil --load-state/--dump-state/--state flags recently, and now my deploy scripts are broken. I think the main problem is block-header information is now also dumped to the state (verified that it isn't on https://github.com/foundry-rs/foundry/releases/tag/nightly-293fad73670b7b59ca901c7f2105bf7a29165a90 https://github.com/foundry-rs/foundry/releases/tag/nightly-5b7e4cb3c882b28f3c32ba580de27ce7381f415a, so I was probably using a version before this).

Seems like the block-header information is loaded, but in an incorrect way, because the block-number is high, but the blocks are not available other than the genesis block. Here's how to repro:

  1. anvil --dump-state state.json
  2. in another terminal, cast send 0x3347f1B5fE96332fEb50ec2B6D5fb5fca7997C30 --value 1wei --private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
  3. close the anvil terminal
  4. anvil --load-state state.json
  5. in another terminal, cast block-number returns 1, but cast block 1 returns this error
Error: 
block Number(Number(1)) not found

whereas cast block 0 works.

@samlaf samlaf added the T-bug Type: bug label Mar 26, 2024
@grandizzy
Copy link
Collaborator

grandizzy commented Mar 27, 2024

this happens because storage hashes are not dumped / reloaded from state file

BlockNumber::Number(num) => *storage.hashes.get(&U64::from(num))?,

@mattsse would be ok to include them?

le: it will also require persisting blocks which cannot be deserialized out of the box....

@samlaf
Copy link
Contributor Author

samlaf commented Apr 10, 2024

Bump. Still seeing this error which is blocking us from switching to anvil (which we would love to now that the RLP encoding fix has unblocked us):

2024-04-10T05:07:05.562591Z ERROR foundry_evm_core::fork::init: It looks like you're trying to fork from an older block with a non-archive node which is not supported. Please try to change your RPC url to an archive node if the issue persists.
Error: 
Could not instantiate forked environment with fork url: http://localhost:8545

Context:
- Failed to get block for block number: 64
latest block number: 64

@samlaf
Copy link
Contributor Author

samlaf commented Apr 26, 2024

Any updates on this? It looks like it should be a relatively simple fix?

@sambacha
Copy link
Contributor

block 0 is technically the correct number for the genesis block

any update @mattsse

@mattsse
Copy link
Member

mattsse commented Apr 27, 2024

It looks like it should be a relatively simple fix?

what's the fix here?

blocks are not exported, just the state, should they be included?

@samlaf
Copy link
Contributor Author

samlaf commented Apr 28, 2024

@mattsse was just carelessly saying.. my bad; I have never actually dived into the anvil codebase.
Just assumed this was "simple" from https://x.com/gakonst/status/1775266582966788549

We have all this info and dumping more stuff is a simple job, just didn't know what people need!

So was assuming you guys would soon be able to dump the block info as well as events (see my other #5906 )

But not sure why @grandizzy was saying blocks can't be deserialized out of the box?

@mattsse
Copy link
Member

mattsse commented Apr 29, 2024

I guess we should consider dumping all blocks as well?

should be possible, and yes "just didn't know what people need!"

@dalechyn
Copy link

dalechyn commented May 1, 2024

Assuming this to be an issue in my case as well.

My goal is to have a persisted anvil setup to be served along with Otterscan, basically creating a persisted on-premise testnet.

On the first launch, when there is no state to load, everything works fine, but after the restart, transaction history (and the block history) can't be loaded. In the image attached below you can see an account having a non-10,000 ETH balance (since it broadcasted some transactions, but they never loaded).

Screenshot 2024-05-01 at 14 18 58

Assuming that if this issue gets fixed, Otterscan would be able to read block data properly.

@dalechyn
Copy link

dalechyn commented May 7, 2024

@mattsse, friendly bump 👉💙

@samlaf
Copy link
Contributor Author

samlaf commented Jun 14, 2024

@mattsse opened a PR with something working. Let me know what you think.

arthurgousset added a commit to celo-org/celo-monorepo that referenced this issue Jul 8, 2024
Starting L1 from scratch instead of JSON state to circumvent this Anvil bug foundry-rs/foundry#7502.
arthurgousset added a commit to celo-org/celo-monorepo that referenced this issue Jul 12, 2024
* refactor(scripts/foundry): moves `run_e2e_tests...sh` file

Small house-keeping change to keep Foundry-related bash scripts organised in a single directory. This script doesn't seem to be called from anywhere, and doesn't have a yarn command in `package.json` so it was fine to move it without any changes in other files.

* feat(package.json): adds yarn commands for e2e and integration tests

Two helper commands to run two Foundry-related bash scripts during local development.

* refactor(migrations_sol): groups imports for better readability

* feat(scripts/foundry): adds MVP L2 migration script

Successfully deploys bytecode to `proxyAdminAddress`.
Does not yet dump state correctly, or call functions to active L2 correctly.

* feat(package.json): adds two helper yarn commands

1. `anvil-devchain:start-L2`: which creates and migrates an L2 devchain
2. `anvil-devchain:check-is-running`: which can be used to check that an anvil instance is running locally and serving at localhost:8546

* chore(migrations_sol): fix typo in function name

* chore(migrations_sol): nit simplifies code comment for better readability

* feat(scripts/foundry): activates CeloDistributionSchedule

Now successfully activates CeloDistributionSchedule.
Does not yet dump state correctly

* refactor(scripts/foundry): moves variables to `constants.sh`

Simplifies maintenance and readability of scripts.

* refactor(scripts/foundry): moves more variables to `constants.sh`

Simplifies maintenance and readability of scripts.

* refactor(constants.sh): read bytecode from Foundry instead of Truffle

* chore(scripts/foundry): adds TODO comment

At this point, the migration works as intended and the L2 is activated.
But, I haven't yet saved the devchain state file.
I'll need to refactor the way the L1 and L2 move the state file around.

* refactor(scripts/foundry): rename to `create_and_migrate_anvil_l2_devchain.sh`

To keep it consistent with the L1 script

* style(migrations_sol): linting

* chore(scripts/foundry): remove debugging statements

* refactor(constants): adds L1 and L2 state file names

* feat(scripts/foundry): successfully dumps L2 state

* refactor(constants): uses `$ANVIL_RPC_URL` instead of hardcoded URL

* refactor(constants): moves libraries to constants

* fix(constants): adds `$PWD` to prevent errors on CI

On CI it seems like `jq` doesn't find the artifacts without an explicit path

```sh
./scripts/foundry/create_and_migrate_anvil_devchain.sh
  shell: /usr/bin/bash -e {0}
  env:
    FOUNDRY_CACHE_KEY: 2
    SUPPORTED_FOUNDRY_VERSION: nightly-f625d0fa7c51e65b4bf1e8f7931cd1c6e2e285e9
    ANVIL_PORT: 8546
jq: error: Could not open file build/contracts/Registry.json: No such file or directory
jq: error: Could not open file build/contracts/Proxy.json: No such file or directory
```

Source: https://github.com/celo-org/celo-monorepo/actions/runs/9808280627/job/27083755032#step:18:9

* chore(workflows): debugging `jq` error on CI

Can't find build artifacts on CI, so trying to work out what might be wrong.

* fix(constants): moves bytecode operation back into scripts

Since the constants are read before any foundry compilation occured, the constants can't read from the Foundry artifacts. Locally, I missed this because I had existing artifacts from previous compilations.

* chore(workflows): debugging `jq` CI error

* fix(scripts/foundry): uses Foundry `out/` directory

Mistakenly read artifacts from Truffle's `build/` directory. Took a while to remember that Foundry write artifacts to `out/`.

Source: https://book.getfoundry.sh/reference/forge/forge-build#build-modes

* chore(scripts/foundry): removes debugging statement

* feat(package.json): updates yarn command names

* docs(CONTRIBUTING): updates yarn command docs

* chore(workflows): add back NPM install step

Not confident enough to remove it in this PR.

* chore(workflows): removes code comment

* feat(workflows): publishes L1 and L2 state

Also updates integration test to explicitly run against L1 devchain, and not L2 devchain.

* fix(scripts/foundry): remove unnecessary `stop_anvil` call

CI doesn't like `lsof` command in `stop_anvil.sh`

```sh
/runner/_work/celo-monorepo/celo-monorepo/packages/protocol/scripts/foundry/stop_anvil.sh: line 10: lsof: command not found
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
```

Source: https://github.com/celo-org/celo-monorepo/actions/runs/9811181588/job/27092901761?pr=11107#step:19:7073

* refactor(workflows): replace CI env with variable from `constants.sh`

Simplifies code and makes it easier to update ports in a single place.

* fix(workflows): adds back `stop_anvil.sh` and installs `lsof` explicitly

CI fails if I don't explicitly kill an existing anvil server, so I decided to install `lsof` which is used by `stop_anvil.sh` to kill processes.

```sh
error binding to 127.0.0.1:8546: error creating server listener: Address already in use (os error 98)
```

Source: https://github.com/celo-org/celo-monorepo/actions/runs/9811346558/job/27093342967?pr=11107#step:20:30

* chore(scripts/foundry): removes code comment

* fix(workflows): install `lsof` in the same step

Not sure if that's the bug, but the last run where the installation and use was in different steps didn't work.

* fix(workflows): hacky fix to run L2 anvil

I'm struggling to terminate the previous anvil server and re-start a new anvil server at the same address (local host 8546).

I tried to work around cumbersome CI environment issues (`lsof` which we use to identify the process is not available on CI).

For now, as a quick fix, I'll start anvil at an arbitrary different port 8547 and run the test against that localhost 8547.

I'll need to refactor this better.

* refactor(scripts/foundry): simplifying `lsof` query to find PID

* fix(workflows): try to terminate anvil server again

* fix(workflows): hacky workaround to prevent Anvil bug when loading state

Starting L1 from scratch instead of JSON state to circumvent this Anvil bug foundry-rs/foundry#7502.

* Update .github/workflows/protocol-devchain-anvil.yml

Co-authored-by: soloseng <[email protected]>

* Update .github/workflows/protocol-devchain-anvil.yml

Co-authored-by: soloseng <[email protected]>

* feat(constants): set CeloDistributionSchedule initial balance to 700M CELO

Arbitrary amount chosen to be approximately equal to `GoldToken.totalSupply()` on the L1 Mainnet (695,313,643 CELO as of this commit). During the real L2 genesis, the VM will calculate and set an appropriate balance.

```sh
# Get address of GoldToken
$ cast call \
0x000000000000000000000000000000000000ce10 \
"getAddressForStringOrDie(string calldata identifier)(address)" \
"GoldToken" \
--rpc-url https://forno.celo.org
0x471EcE3750Da237f93B8E339c536989b8978a438

# Call `totalSupply()` on `GoldToken.sol`
$ cast call \
0x471EcE3750Da237f93B8E339c536989b8978a438 \
"totalSupply()(uint256)" \
--rpc-url https://forno.celo.org
695313643195035937058427065 [6.953e26]

# Convert units from wei to ether
$ cast to-unit 695313643195035937058427065 ether
695313643.195035937058427065
```

* docs(README): improves README

More context, examples, background, files in the package.

---------

Co-authored-by: soloseng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-bug Type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants