feat(foundry): publish L2 state in @celo/devchain-anvil

[arthurgousset]

Description

1. adds script (create_and_migrate_anvil_l2_devchain.sh) to create, migrate, and dump L1 and L2 devchain state to distinct JSON files (devchain.json and l2-devchain.json respectively).
2. updates NPM publishing script to release L1 and L2 devchain state to @celo/devchain-anvil.

Other changes

1. refactors Foundry scripts to use constants defined in new constants.sh file
2. adds and updates various yarn commands:

   $ yarn anvil-devchain:start-L1
   $ yarn anvil-devchain:start-L2
   $ yarn anvil-devchain:e2e-tests
   $ yarn anvil-devchain:integration-tests
   $ yarn anvil-devchain:status

3. moves run_e2e_tests_in_anvil.sh file to the scripts/foundry/ directory with other Foundry-related scripts.
4. groups contract imports in Migrations.s.sol by Solidity version for better readability
5. small typo fixes

Tested

Tested L2 migration script ✅

$ yarn anvil-devchain:start-L2
# ...
Activating L2 on the devchain...
anvil_setCode
null
✨  Done in 10.90s.

(in a separate terminal) Check that proxyAdminAddress has bytecode ✅

$ cast rpc eth_getCode \
0x4200000000000000000000000000000000000018 \
latest \
--rpc-url=http://127.0.0.1:8546

"0x608060405234801561001057600080fd5b506...

(in a separate terminal) Check that isL2() is true on CeloDistributionSchedule.sol. ✅
For context, we can call isL2() on CeloDistributionSchedule.sol and other core contracts because they inherit from IsL2Check.sol

# Get address of CeloDistributionSchedule
$ cast call \
0x000000000000000000000000000000000000ce10 \
"getAddressForStringOrDie(string calldata identifier)(address)" \
"CeloDistributionSchedule" \
--rpc-url=http://127.0.0.1:8546

0xA16cF67AFa80BB9Ce7a325597F80057c6B290fD4

# Call `isL2()` on `CeloDistributionSchedule.sol`
cast call \
0xA16cF67AFa80BB9Ce7a325597F80057c6B290fD4 \
"isL2()(bool)" \
--rpc-url=http://127.0.0.1:8546
true

Tested each new helper command locally from the `packages/protocol` directory ✅

$ yarn anvil-devchain:e2e-tests

Generating and running devchain before running e2e tests...
# ...
Ran 1 test suite in 172.54ms (10.86ms CPU time): 1 tests passed, 0 failed, 0 skipped (1 total tests)
Stopping devchain...
Connection to localhost port 8546 [tcp/*] succeeded!
Killed Anvil
✨  Done in 391.47s.

$ yarn anvil-devchain:integration-tests

Generating and running devchain before running integration tests...
# ...
Ran 1 test suite in 375.34ms (343.75ms CPU time): 2 tests passed, 0 failed, 0 skipped (2 total tests)
Stopping devchain...
Connection to localhost port 8546 [tcp/*] succeeded!
Killed Anvil
✨  Done in 397.19s.

$ yarn anvil-devchain:check-is-running
# ...
Connection to localhost port 8546 [tcp/*] succeeded!
✨  Done in 0.05s.

Related issues

• Fixes Support devchain with L2 already transitioned #11103

Backwards compatibility

Yes, existing L1 devchain is unchanged, only adding a new L2 devchain state JSON.

Documentation

The set of community facing docs that have been added/modified because of this change

[openzeppelin-code]

feat: publish L2 state for @celo/devchain-anvil

Generated at commit: 7ac73f7d991609494dbea3a51ba3782284644b9b

🚨 Report Summary

	Severity Level	Results
Contracts	Critical High Medium Low Note Total	2 3 0 15 41 61
Dependencies	Critical High Medium Low Note Total	0 0 0 0 0 0

---

For more details view the full report in OpenZeppelin Code Inspector

[gitguardian]

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
10538986	Triggered	Generic High Entropy Secret	1b90853	packages/protocol/scripts/foundry/constants.sh	View secret
10538986	Triggered	Generic High Entropy Secret	4a7606b	packages/protocol/scripts/foundry/constants.sh	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[arthurgousset]

(for future reference)

This PR implements a workaround (in a906048) to prevent a known Anvil bug when loading state.

Basically, I'm generating and running an L1 devchain from scatch, and use that to run the integration test here:

celo-monorepo/.github/workflows/protocol-devchain-anvil.yml

Lines 127 to 146 in a906048

	# Starting L1 from scratch instead of JSON state to circumvent this Anvil bug https://github.com/foundry-rs/foundry/issues/7502
	# Install `lsof` dependency, because it's not readily available on CI, but is required by
	# `create_and_migrate_anvil_l2_devchain.sh`, because it uses `stop_anvil.sh` to kill
	# existing anvil servers.
	- name: Run integration tests against L1 devchain
	if: success() || failure()
	run: |
	sudo apt-get update
	sudo apt-get install -y lsof
	source ./scripts/foundry/constants.sh
	echo "Starting L1 from scratch to circumvent Anvil bug"
	source ./scripts/foundry/create_and_migrate_anvil_devchain.sh
	forge test -vvv \
	--match-path "test-sol/integration/*" \
	--fork-url $ANVIL_RPC_URL
	./scripts/foundry/stop_anvil.sh

This takes longer than simply starting an L1 devchain from JSON state, but prevents the Anvil bug when loading state.

A better approach would be to start an anvil instance using JSON state like here:

celo-monorepo/.github/workflows/protocol-devchain-anvil.yml

Lines 138 to 154 in 28464b6

	- name: Run integration tests against L1 devchain
	if: success() || failure()
	run: |
	source ./scripts/foundry/constants.sh
	anvil \
	--port 8547 \
	--state $TMP_FOLDER/$L1_DEVCHAIN_FILE_NAME \
	--state-interval $STATE_INTERVAL \
	--gas-limit $GAS_LIMIT \
	--code-size-limit $CODE_SIZE_LIMIT \
	--balance $BALANCE \
	--steps-tracing &
	forge test -vvv \
	--match-path "test-sol/integration/*" \
	--fork-url http://127.0.0.1:8547

Anvil bug

Known issue:

• recent anvil --state/dump-state changes are broken foundry-rs/foundry#7502
• forge script: "Could not instantiate forked environment with fork url" --rpc-url to local anvil chain foundry-rs/foundry#7952
• Forge fails to fork against Anvil instances that load state foundry-rs/foundry#7284

The issue (foundry-rs/foundry#7502) is marked as fixed by this PR:

• fix(anvil): block dumps foundry-rs/foundry#8160

Since we're not on the latest Foundry nightly, we won't be able to benefit from this PR until we bump our version.

[martinvol]

is there a script within this PR where you can import a L1 state and it'll dump the state after the L2?

[arthurgousset]

is there a script within this PR where you can import a L1 state and it'll dump the state after the L2?

@martinvol: Good question, not in the current state.

There is a limitation in Foundry that prevents us from loading state, and then dumping state to a different file. For example:

$ anvil \
--port 8546 \
--state ~/Downloads/devchain-11107-2024-07-08/devchain.json \
--dump-state ~/Downloads/l2-devchain.json

error: the argument '--state <PATH>' cannot be used with '--dump-state <PATH>'

But, this was a design I considered during this PR. We can circumvent this restriction by copying and renaming files during runtime like we do for L1:

celo-monorepo/packages/protocol/scripts/foundry/create_and_migrate_anvil_devchain.sh

Lines 67 to 68 in ff291e0

	# Rename devchain artifact and remove unused directory
	mv $ANVIL_FOLDER/state.json $TMP_FOLDER/$L1_DEVCHAIN_FILE_NAME

But, I took the view that it was overkill because it also required refactoring start_anvil.sh which specifies the name of the JSON state file. That's because we want two JSON files, L1 and L2.

celo-monorepo/packages/protocol/scripts/foundry/start_anvil.sh

Lines 23 to 26 in ff291e0

	# Start anvil
	anvil \
	--port $ANVIL_PORT \
	--dump-state $ANVIL_FOLDER \

Definitely doable, but for the scope of this PR, which was to release an L2 devchain, I found the current approach to be the fastest path to an L2 devchain without significantly re-organising our scripts.

We can refactor our bash and Foundry scripts to work differently if useful.