Tests fail on second run with No module named 'setuptools.tests'

[jaraco]

Running tests on Python 3.7, I can run tox and the tests run fine.

If I try to run them a second time, the tests fail early when pytest tries to load plugins as defined in conftest.py and setuptools.tests doesn't exist.

It seems that it doesn't exist because sometime during the first test run, the "develop" install of setuptools has been replaced with an regular install of setuptools, and since "tests" aren't installed in a regular install, those modules aren't available.

[benoit-pierre]

I can't reproduce this.

[jaraco]

Since it appeared recently, it may be related to recent releases of tox or pytest or (more likely) the private release of tox-venv that I'm running from https://github.com/jaraco/tox-venv/tree/feature/update-env-installer.

[benoit-pierre]

I still can't reproduce, using:

tox==3.4.0
tox-venv==0.3.1 # installed from https://github.com/jaraco/tox-venv/archive/feature/update-env-installer.zip

And the tox env is using:

apipkg==1.5
atomicwrites==1.2.1
attrs==18.2.0
contextlib2==0.5.5
coverage==4.5.1
execnet==1.5.0
flake8==3.5.0
importlib-metadata==0.5
mccabe==0.6.1
mock==2.0.0
more-itertools==4.3.0
path.py==11.3.0
Paver==1.3.4
pbr==4.2.0
pluggy==0.7.1
py==1.6.0
pycodestyle==2.3.1
pyflakes==1.6.0
pytest==3.8.1
pytest-cov==2.6.0
pytest-fixture-config==1.3.0
pytest-flake8==1.0.2
pytest-shutil==1.3.0
pytest-virtualenv==1.3.0
six==1.11.0
termcolor==1.1.0
virtualenv==16.0.0

[jaraco]

Confirmed; if I back out to use tox-venv 0.3.1, the problem goes away.

[benoit-pierre]

Nevermind, pip behaviour is really confusing in that case: it built a wheel for tox-venv, but did not update...

[benoit-pierre]

This time I made sure tox-venv is up-to-date, and I still can't reproduce.

[benoit-pierre]

Are you doing all of this from a virtualenv/venv?

[jaraco]

@benoit-pierre The failure mode is only when you have the yet-unreleased code from tox-dev/tox-venv#25. You can probably replicate the issue with pip install -i https://m.devpi.net/jaraco/dev -U tox-venv.

[jaraco]

Both tox and tox-venv have to be installed outside of ./tox; I happen to be using my system site-packages for that installation location.

[benoit-pierre]

Yes, I pip installed from https://github.com/jaraco/tox-venv/archive/feature/update-env-installer.zip.

[jaraco]

hmm. I'm now unable to replicate the issue either.

[jaraco]

I'm struggling now to think of what state that I may have been in that caused the issue. Unless it pops up again, I'll close this issue, and if it does, I'll be more careful about capturing the state before making changes.

[benoit-pierre]

I think you can get into this state if you interrupt the develop phase, and then run tox again.