build with setup.py errors with 60.9.1 [BUG]

[C-nit]

setuptools version

setuptools==60.9.1

Python version

Python 3.7.6

OS

Linux

Additional environment information

This was encountered on TravisCI using the pypi deployment provider, happens with multiple linux versions from precise to xenial.
Not tested on Linux outside of Travis. Cannot reproduce it on MacOS.

Description

Running python setup.py sdist bdist_wheel errors with newest version of setuptools errors.
Fixed by using version previous to 60.9.

This might be related to #3113 as the error occurs at setuptools/_vendor/importlib_metadata/__init__.py

Expected behavior

dist/* to be built without errors

How to Reproduce

1. Have a python package using setup.py calling setup()
2. Have TravisCI build distributions during it's deploy step

deploy:
  provider: pypi
  distributions: "sdist bdist_wheel"
  username: ${PYPI_USERNAME}
  password: ${PYPI_PASSWORD}

that boils down to the following commands that can be reproduced

script:
  # mimic travis deploy steps concerning build during deployment step
  # https://github.com/travis-ci/dpl/blob/v1.10.16/lib/dpl/provider/pypi.rb
  # https://github.com/travis-ci/dpl/blob/b577682ac17ff10b7f3fe58c5da1113b05e68cb7/lib/dpl/provider.rb#L187
  - |
    if [ -z ${VIRTUAL_ENV+x} ]; then export PIP_USER=yes; fi && \
    wget -nv -O - https://bootstrap.pypa.io/get-pip.py | python - --no-setuptools --no-wheel && \
    pip install --upgrade --upgrade-strategy eager setuptools twine wheel
  - python setup.py sdist bdist_wheel

This can be fixed by reverting to version previous to 60.9.
For example on travis they provide a parameter:

deploy:
  provider: pypi
  distributions: "sdist bdist_wheel"
  username: ${PYPI_USERNAME}
  password: ${PYPI_PASSWORD}
  setuptools_version: "60.8.2"

Output

Deploying application
Traceback (most recent call last):
  File "setup.py", line 8, in <module>
    setup()
  File "/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools/__init__.py", line 155, in setup
    return distutils.core.setup(**attrs)
  File "/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 109, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools/dist.py", line 462, in __init__
    for k, v in attrs.items()
  File "/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 293, in __init__
    self.finalize_options()
  File "/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools/dist.py", line 850, in finalize_options
    for ep in sorted(loaded, key=by_order):
  File "/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools/dist.py", line 849, in <lambda>
    loaded = map(lambda e: e.load(), filtered)
  File "/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 196, in load
    return functools.reduce(getattr, attrs, module)
AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'

[C-nit]

also setuptools==60.9.0 and setuptools==60.9.2

[lorengordon]

Yep, we've been seeing the same behavior with 60.9. Will continue to remain pinned to 60.8.2 for now.

[jaraco]

A similar issue was reported in #2029. In that case, the problem came from inconsistencies found between two different versions of 'setuptools'. That hook was added to Setuptools around v42 and removed in c719690 for v46. So my guess is there's another copy of setuptools on the path from which the metadata is taking precedence.

[jaraco]

I thought maybe I could replicate the issue with something like this:

from ubuntu:focal
run apt update
run apt upgrade -y
run apt install -y python3-distutils wget
run ln -s python3 /usr/bin/python
run wget https://bootstrap.pypa.io/get-pip.py -O - | python - --no-setuptools --no-wheel
run pip install 'setuptools<46' virtualenv
run virtualenv --system-site-packages env
run env/bin/pip install --upgrade setuptools wheel
cmd env/bin/python -c "__import__('setuptools').setup()" sdist

or maybe

from ubuntu:focal
run apt update
run apt upgrade -y
run apt install -y python3-distutils wget
run ln -s python3 /usr/bin/python
run wget https://bootstrap.pypa.io/get-pip.py -O - | python - --no-setuptools --no-wheel
run pip install 'setuptools<46' virtualenv
run pip install --user --upgrade setuptools wheel
cmd python -c "__import__('setuptools').setup()" sdist

But in both cases, the error did not occur.

I did not try ubuntu xenial, because it comes with a Python that's no longer supported by the latest Setuptools.

Can someone help put together a Dockerfile that can replicate the issue?

[lorengordon]

Well it's not a Dockerfile, but it is reproducible in our CI config, and it is isolated to the bump of setuptools... plus3it/watchmaker#1958

Fwiw, it also seems specific to python 3.7 and 3.8.

[lorengordon]

Of course now that I look at it more closely, I see it being a failure in running virtualenv --version, and the traceback and travis output was throwing me off at first glance. That's using virtualenv v20.9.0. So maybe some related versioning dependency between that and setuptools...

[jaraco]

I appreciate that it's reproducible in your CI environment. Unfortunately, unless we can somehow ascertain the factors that make it fail in CI and reproduce those in a local environment, it will be near impossible to diagnose. I do suspect there's something about having two different versions of setuptools installed, on of which is >42,<46. I just yet don't know what other factor(s) are relevant.

[lorengordon]

Oh I totally understand. I didn't mean to imply that it was somehow on you to figure out from our setup! 😆 It's been failing for a bit, but I've only just started looking into it (pinning CI dependencies makes it not critical for us!). I'll post here if I figure out how to reproduce more simply.

[lorengordon]

The mentions of focal and xenial above got me to check our travis-ci config, and it was still using bionic. Updated that to focal and the error went away. So makes sense to me that the interaction of the older bionic worker and its bundled packages, and our pinned virtualenv and setuptools versions resulted in the error. Anyway, sorry for the noise, but hopefully that's enough of a clue to help someone else that ends up here.

[C-nit]

Sorry this took me a while.

In the environment I encountered the issue, that is Travis CI, we are automatically running in a virtualenv.
Deactivating this virtualenv indeed shows there is a setuptools==44.1.1 apparently in a python2 system environment

$ pip list
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Package    Version
---------- ----------
bzr        2.8.0.dev1
configobj  5.0.6
mercurial  4.8
pip        20.3.4
setuptools 44.1.1
six        1.11.0
virtualenv 15.1.0
wheel      0.37.1

[jaraco]

That's interesting. I wouldn't expect a Python 3 virtualenv to be finding metadata for a version of Setuptools found in Python 2. Can you run this in the failing environment:

from setuptools._importlib import metadata as md
print([(dist.name, dist._path) for dist in md.distributions() if 'setuptools' in dist.name.lower()])

[C-nit]

So you were right, nothing to do with python2.

Running

$ python -c "from setuptools._importlib import metadata as md; print([(dist.name, dist._path) for dist in md.distributions() if 'setuptools' in dist.name.lower()])"

at 2 times:

1. in the virtualenv, with setuptools==60.5.0

pip                  21.3.1
importlib-metadata   1.3.0
setuptools           60.5.0

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools._importlib'

2. in the virtualenv, updated to setuptools==60.9.2

On the update, following TravisCI's steps on the deploy phase, we get a new version of pip.

$ if [ -z ${VIRTUAL_ENV+x} ]; then export PIP_USER=yes; fi && \
wget -nv -O - https://bootstrap.pypa.io/get-pip.py | python - --no-setuptools --no-wheel && \
pip install --upgrade --upgrade-strategy eager "setuptools==60.9.2" twine wheel

there is actually a dependency conflict for importlib-metadata on this upgrade that I don't think is relevant (flake8 is not the problem):

flake8 4.0.1 requires importlib-metadata<4.3; python_version < "3.8", but you have importlib-metadata 4.11.1 which is incompatible.

the versions are like this:

pip                  22.0.3
importlib-metadata   4.11.1
setuptools           60.9.2

[('setuptools', PosixPath('/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools-42.0.2.dist-info')), ('setuptools', PosixPath('/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools-60.9.2.dist-info'))]

so I reran the build looking into the sitepackages

$ ls -lt /home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools*

1. in the virtualenv, with setuptools==60.5.0

/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools:
total 1024
drwxr-xr-x 4 travis travis   4096 Jan 25 07:56  _distutils
drwxrwxr-x 2 travis travis   4096 Jan 25 07:56  __pycache__
drwxr-xr-x 3 travis travis     25 Jan 25 07:56  tests
drwxrwxr-x 5 travis travis    139 Jan 25 07:56  _vendor
-rw-rw-r-- 1 travis travis   7077 Jan 25 07:56  archive_util.py
-rw-rw-r-- 1 travis travis  10536 Jan 25 07:56  build_meta.py
-rw-rw-r-- 1 travis travis  65536 Jan 25 07:56  cli-32.exe
-rw-rw-r-- 1 travis travis  74752 Jan 25 07:56  cli-64.exe
-rw-r--r-- 1 travis travis 137216 Jan 25 07:56  cli-arm64.exe
-rw-rw-r-- 1 travis travis  65536 Jan 25 07:56  cli.exe
-rw-rw-r-- 1 travis travis  23153 Jan 25 07:56  config.py
-rw-rw-r-- 1 travis travis   5499 Jan 25 07:56  depends.py
-rw-rw-r-- 1 travis travis    218 Jan 25 07:56  _deprecation_warning.py
-rw-rw-r-- 1 travis travis    949 Jan 25 07:56  dep_util.py
-rw-rw-r-- 1 travis travis  43154 Jan 25 07:56  dist.py
-rw-rw-r-- 1 travis travis   1555 Jan 25 07:56  errors.py
-rw-rw-r-- 1 travis travis   1684 Jan 25 07:56  extension.py
-rw-rw-r-- 1 travis travis   4873 Jan 25 07:56  glob.py
-rw-rw-r-- 1 travis travis  65536 Jan 25 07:56  gui-32.exe
-rw-rw-r-- 1 travis travis  75264 Jan 25 07:56  gui-64.exe
-rw-r--r-- 1 travis travis 137728 Jan 25 07:56  gui-arm64.exe
-rw-rw-r-- 1 travis travis  65536 Jan 25 07:56  gui.exe
-rw-rw-r-- 1 travis travis   2392 Jan 25 07:56  _imp.py
-rw-rw-r-- 1 travis travis   7494 Jan 25 07:56  __init__.py
-rw-rw-r-- 1 travis travis   3824 Jan 25 07:56  installer.py
-rw-rw-r-- 1 travis travis    812 Jan 25 07:56  launch.py
-rw-r--r-- 1 travis travis    863 Jan 25 07:56  logging.py
-rw-rw-r-- 1 travis travis   5217 Jan 25 07:56  monkey.py
-rw-rw-r-- 1 travis travis  50561 Jan 25 07:56  msvc.py
-rw-rw-r-- 1 travis travis   3093 Jan 25 07:56  namespaces.py
-rw-rw-r-- 1 travis travis  40092 Jan 25 07:56  package_index.py
-rw-rw-r-- 1 travis travis    245 Jan 25 07:56  py34compat.py
-rw-rw-r-- 1 travis travis  14348 Jan 25 07:56  sandbox.py
-rw-rw-r-- 1 travis travis    218 Jan 25 07:56 'script (dev).tmpl'
-rw-rw-r-- 1 travis travis    138 Jan 25 07:56  script.tmpl
-rw-rw-r-- 1 travis travis    941 Jan 25 07:56  unicode_utils.py
-rw-rw-r-- 1 travis travis    144 Jan 25 07:56  version.py
-rw-rw-r-- 1 travis travis   8[274]() Jan 25 07:56  wheel.py
-rw-rw-r-- 1 travis travis    714 Jan 25 07:56  windows_support.py
drwxrwxr-x 3 travis travis   4096 Dec 21  2019  command
drwxrwxr-x 3 travis travis     44 Dec 21  2019  extern
-rw-rw-r-- 1 travis travis   2013 Dec 21  2019  lib2to3_ex.py
-rw-rw-r-- 1 travis travis   1496 Dec 21  2019  py27compat.py
-rw-rw-r-- 1 travis travis    838 Dec 21  2019  py31compat.py
-rw-rw-r-- 1 travis travis   1330 Dec 21  2019  py33compat.py
-rw-rw-r-- 1 travis travis   [230]()2 Dec 21  [201]()9  site-patch.py
-rw-rw-r-- 1 travis travis   8493 Dec 21  2019  ssl_support.py
/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools-60.5.0.dist-info:
total 52
-rw-r--r-- 1 travis travis     4 Jan 25 07:56 INSTALLER
-rw-r--r-- 1 travis travis 23852 Jan 25 07:56 RECORD
-rw-r--r-- 1 travis travis  2636 Jan 25 07:56 entry_points.txt
-rw-r--r-- 1 travis travis  1050 Jan 25 07:56 LICENSE
-rw-r--r-- 1 travis travis  5936 Jan 25 07:56 METADATA
-rw-r--r-- 1 travis travis    41 Jan 25 07:56 top_level.txt
-rw-r--r-- 1 travis travis    92 Jan 25 07:56 WHEEL
/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools-42.0.2.dist-info:
total 48
-rw-rw-r-- 1 travis travis     4 Dec 21  2019 INSTALLER
-rw-rw-r-- 1 travis travis 14419 Dec 21  2019 RECORD
-rw-rw-r-- 1 travis travis   239 Dec 21  2019 dependency_links.txt
-rw-rw-r-- 1 travis travis  3[206]() Dec 21  2019 entry_points.txt
-rw-rw-r-- 1 travis travis  1078 Dec 21  2019 LICENSE
-rw-rw-r-- 1 travis travis  3746 Dec 21  2019 METADATA
-rw-rw-r-- 1 travis travis    38 Dec 21  2019 top_level.txt
-rw-rw-r-- 1 travis travis   110 Dec 21  2019 WHEEL
-rw-rw-r-- 1 travis travis     1 Dec 21  2019 zip-safe

No changes to the virtual environment by me btw, this comes like this from travis 🤯

2. in the virtualenv, updated to setuptools==60.9.2

/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools:
total 1044
drwxr-xr-x 2 travis travis      6 Feb 23 12:12  tests
drwxrwxr-x 8 travis travis    249 Feb 23 12:12  _vendor
drwxrwxr-x 2 travis travis   4096 Feb 23 12:12  __pycache__
drwxr-xr-x 3 travis travis     44 Feb 23 12:12  extern
drwxr-xr-x 4 travis travis   4096 Feb 23 12:12  _distutils
drwxrwxr-x 3 travis travis   4096 Feb 23 12:12  command
-rw-r--r-- 1 travis travis  65536 Feb 23 12:12  gui.exe
-rw-r--r-- 1 travis travis   3824 Feb 23 12:12  installer.py
-rw-r--r-- 1 travis travis    812 Feb 23 12:12  launch.py
-rw-r--r-- 1 travis travis   1148 Feb 23 12:12  logging.py
-rw-r--r-- 1 travis travis   5217 Feb 23 12:12  monkey.py
-rw-r--r-- 1 travis travis  50561 Feb 23 12:12  msvc.py
-rw-r--r-- 1 travis travis   3093 Feb 23 12:12  namespaces.py
-rw-r--r-- 1 travis travis  40055 Feb 23 12:12  package_index.py
-rw-r--r-- 1 travis travis    245 Feb 23 12:12  py34compat.py
-rw-r--r-- 1 travis travis  14348 Feb 23 12:12  sandbox.py
-rw-r--r-- 1 travis travis    218 Feb 23 12:12 'script (dev).tmpl'
-rw-r--r-- 1 travis travis    138 Feb 23 12:12  script.tmpl
-rw-r--r-- 1 travis travis    941 Feb 23 12:12  unicode_utils.py
-rw-r--r-- 1 travis travis    144 Feb 23 12:12  version.py
-rw-r--r-- 1 travis travis   8274 Feb 23 12:12  wheel.py
-rw-r--r-- 1 travis travis    714 Feb 23 12:12  windows_support.py
-rw-r--r-- 1 travis travis  65536 Feb 23 12:12  gui-32.exe
-rw-r--r-- 1 travis travis  75264 Feb 23 12:12  gui-64.exe
-rw-r--r-- 1 travis travis 137728 Feb 23 12:12  gui-arm64.exe
-rw-r--r-- 1 travis travis 137216 Feb 23 12:12  cli-arm64.exe
-rw-r--r-- 1 travis travis  65536 Feb 23 12:12  cli.exe
-rw-r--r-- 1 travis travis  23153 Feb 23 12:12  config.py
-rw-r--r-- 1 travis travis   5499 Feb 23 12:12  depends.py
-rw-r--r-- 1 travis travis    949 Feb 23 12:12  dep_util.py
-rw-r--r-- 1 travis travis  44087 Feb 23 12:12  dist.py
-rw-r--r-- 1 travis travis   1555 Feb 23 12:12  errors.py
-rw-r--r-- 1 travis travis   1675 Feb 23 12:12  extension.py
-rw-r--r-- 1 travis travis   4873 Feb 23 12:12  glob.py
-rw-r--r-- 1 travis travis   7070 Feb 23 12:12  archive_util.py
-rw-r--r-- 1 travis travis  10902 Feb 23 12:12  build_meta.py
-rw-r--r-- 1 travis travis  65536 Feb 23 12:12  cli-32.exe
-rw-r--r-- 1 travis travis  74752 Feb 23 12:12  cli-64.exe
-rw-r--r-- 1 travis travis    891 Feb 23 12:12  _importlib.py
-rw-r--r-- 1 travis travis    675 Feb 23 12:12  _itertools.py
-rw-r--r-- 1 travis travis    176 Feb 23 12:12  _path.py
-rw-r--r-- 1 travis travis    501 Feb 23 12:12  _reqs.py
-rw-r--r-- 1 travis travis    218 Feb 23 12:12  _deprecation_warning.py
-rw-r--r-- 1 travis travis   1972 Feb 23 12:12  _entry_points.py
-rw-r--r-- 1 travis travis   2392 Feb 23 12:12  _imp.py
-rw-r--r-- 1 travis travis   7461 Feb 23 12:12  __init__.py
-rw-rw-r-- 1 travis travis   2013 Dec 21  2019  lib2to3_ex.py
-rw-rw-r-- 1 travis travis   1496 Dec 21  2019  py27compat.py
-rw-rw-r-- 1 travis travis    838 Dec 21  2019  py31compat.py
-rw-rw-r-- 1 travis travis   [1330]() Dec 21  2019  py33compat.py
-rw-rw-r-- 1 travis travis   2302 Dec 21  2019  site-patch.py
-rw-rw-r-- 1 travis travis   8493 Dec 21  2019  ssl_support.py
/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools-60.9.2.dist-info:
total 60
-rw-r--r-- 1 travis travis 31027 Feb 23 12:12 RECORD
-rw-r--r-- 1 travis travis     0 Feb 23 12:12 REQUESTED
-rw-r--r-- 1 travis travis     4 Feb 23 12:12 INSTALLER
-rw-r--r-- 1 travis travis  2635 Feb 23 12:12 entry_points.txt
-rw-r--r-- 1 travis travis  1050 Feb 23 12:12 LICENSE
-rw-r--r-- 1 travis travis  5932 Feb 23 12:12 METADATA
-rw-r--r-- 1 travis travis    41 Feb 23 12:12 top_level.txt
-rw-r--r-- 1 travis travis    92 Feb 23 12:12 WHEEL
/home/travis/virtualenv/python3.7.6/lib/python3.7/site-packages/setuptools-42.0.2.dist-info:
total 48
-rw-rw-r-- 1 travis travis     4 Dec 21  2019 INSTALLER
-rw-rw-r-- 1 travis travis 14419 Dec 21  2019 RECORD
-rw-rw-r-- 1 travis travis   239 Dec 21  2019 dependency_links.txt
-rw-rw-r-- 1 travis travis  3206 Dec 21  2019 entry_points.txt
-rw-rw-r-- 1 travis travis  1078 Dec 21  2019 LICENSE
-rw-rw-r-- 1 travis travis  3746 Dec 21  2019 METADATA
-rw-rw-r-- 1 travis travis    38 Dec 21  2019 top_level.txt
-rw-rw-r-- 1 travis travis   110 Dec 21  2019 WHEEL
-rw-rw-r-- 1 travis travis     1 Dec 21  2019 zip-safe

[C-nit]

@jaraco is this something we can expect to go away with future versions independent of travis fixing the virtual environment?
Or is this not something you plan to accommodate for?

[mriedem]

I hit something similar with Travis Enterprise using the default (currently ubuntu bionic) image. I had to upgrade the pypi deploy job from python 3.6 (the default in the image) to 3.8 because the travis dpl tooling no longer supports 3.6 (not a surprise). Then I hit this issue. The travis image already had setuptools 60.5.0 installed and then the dpl tooling upgraded to the latest (currently 62.1.0). I'm guessing those mixed versions are somehow interfering with each other, or the pip uninstall of the already installed 60.5.0 didn't clean everything up.

I tried adding an install script hook to upgrade setuptools before the deploy ran but that didn't work - it was basically the same thing as the dpl tooling upgrading setuptools.

The workaround I heard from someone else with the same issue was to pin the pypi deploy in Travis build config to the same version of setuptools that was already in the image using the setuptools_version option. That worked as my workaround.

I'm hoping that maybe when Travis Enterprise supports Ubuntu Focal I can try again without the pinned setuptools version.

[Josue-T]

Hello,

I've the same issue in a venv while I was installing cffi.
It was on ARMv7 on debian bullseye with setuptools from system version 52.0.0-4 and when setuptools in the venv is version 62.3.2.
Downgrading setuptools in the venv to version 60.8.2 fixed the issue on my side.

[mriedem]

Quick follow up on this, I found that building in Travis CI with python 3.9 works around this problem.

[jaraco]

is this something we can expect to go away with future versions independent of travis fixing the virtual environment? Or is this not something you plan to accommodate for?

I'm not confident there's anything Setuptools could do here, and until we have a repro that can be used to locally investigate, we're kinda stuck. Besides some of the workarounds described above, there may be other options, such as writing a routine to clear out manually old versions of setuptools before running build tools.

If we could identify something that could be patched in the current or perhaps a past version of Setuptools that would hepl the situation, I'd consider it, but at this point I've no options to consider.