fix(consume,plugins): Fix suite duplication due to xdist, consume cli refactor

[marioevz]

🗒️ Description

Fixes test_suite fixture from spawning duplicated test suites when xdist is used.

Reason being that xdist spawns a different process per worker to distribute the load, and the session or module scoped fixtures are not really shared between them.

Fix is to use the file system to create global locks per consume session, achieved by creating a temporary folder that is unique to each consume instance and is shared between the xdist processes, but do not disrupt other consume sessions.

The fixture also creates a lock per test-suite, so it is possible to run different test suites (i.e. one for rlp and one for engine) in the same consume instance.

Consume commands were also refactored to reduce code and consume hive command is now available which runs test suites of consume rlp and consume engine simultaneously.

🔗 Related Issues

✅ Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
• Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

[spencer-tb]

Really nice! LGTM.

I'm going create a small PR based on this to split pytest_commands.py to the following within src/cli:

pytest_commands/
├── common.py
├── consume.py
├── fill.py
└── __init__.py
...
├── execute.py
├── fuzzer.py

It should keep things clean when adding execute/fuzzer!

Feel free to merge this or wait for my small refactor :D

[spencer-tb]

The fixture also creates a lock per test-suite, so it is possible to run different test suites (i.e. one for rlp and one for engine) in the same consume instance.

Just so I understand this essentially means running consume hive with parallelism where only one instance is spawned for both rlp and engine?

[marioevz]

The fixture also creates a lock per test-suite, so it is possible to run different test suites (i.e. one for rlp and one for engine) in the same consume instance.

Just so I understand this essentially means running consume hive with parallelism where only one instance is spawned for both rlp and engine?

You can either launch consume rlp and consume engine separately or do consume hive with any xdist number parameter, and the result in hive view should be exactly the same.

Basically we could reduce the code in the hive cluster script.