Second run of tox -e py results in a test error for test marked with no_python2

[Jackenmen]

Describe the bug

no_python2 test errors on main while following Black's docs for contributing.

To Reproduce

Per docs:

pipenv install --dev
pipenv shell
pre-commit install
pre-commit run -a
tox -e py

The above works properly, however another run of tox -e py results in an error due to tox's usage of existing py environment causing no_python2 mark to be fulfilled:

___________________________ BlackTestCase.test_python2_should_fail_without_optional_install ___________________________
[gw2] win32 -- Python 3.8.9 c:\users\jakub\desktop\black\.tox\py\scripts\python.exe

self = <tests.test_black.BlackTestCase testMethod=test_python2_should_fail_without_optional_install>

    @pytest.mark.no_python2
    def test_python2_should_fail_without_optional_install(self) -> None:
        if sys.version_info < (3, 8):
            self.skipTest(
                "Python 3.6 and 3.7 will install typed-ast to work and as such will be"
                " able to parse Python 2 syntax without explicitly specifying the"
                " python2 extra"
            )

        source = "x = 1234l"
        tmp_file = Path(black.dump_to_file(source))
        try:
            runner = BlackRunner()
            result = runner.invoke(black.main, [str(tmp_file)])
>           self.assertEqual(result.exit_code, 123)
E           AssertionError: 0 != 123

C:\Users\Jakub\Desktop\black\tests\test_black.py:468: AssertionError

Expected behavior

No test errors.

Environment (please complete the following information):

• Version: main
• OS and Python version: Windows/Python 3.9.5

Does this bug also happen on main?

Yes

Additional context

Originally posted in #2238 (#2238 (comment))

[MarcoGorelli]

You're right, can confirm this reproduces. $ tox -e py -- -x passes, but if I run it again, it fails