[BUG] Importing pip breaks setuptools

[dwpaley]

setuptools version

setuptools>=60

Python version

tested on python 3.8 and 3.10

OS

tested on MacOS 10.15.7 and CentOS 7.4

Additional environment information

No response

Description

• If I import setuptools, then import pip, then call setuptools.setup, then I get a crash with 'distutils' missing from sys.modules.
• If I import pip, then import setuptools, I get an immediate crash on assert '_distutils' in core.__file__

A few connected issues are:

• configure.py crashes connected to distutils and/or setuptools cctbx/cctbx_project#710
• Setuptools 60 is breaking tests pip#10742
• AssertionError no _distutils when running get-pip with Setuptools and USE_DISTUTILS=local #2993

Expected behavior

Wish I could still use setuptools after importing pip

How to Reproduce

First bug:

1. Install setuptools>=60
2. Create a trivial package and a toml file
3. Create a script repro.py that imports setuptools, then imports pip, then calls setuptools.setup
4. Execute the script with action build

The first bug is not triggered if any of these changes are made:

• Downgrade to setuptools=59
• Rename repro.py to setup.py
• Skip importing pip

Second bug:

1. Install setuptools>=60
2. Interactively or in a script, import pip and then setuptools

Output

First bug:

$ conda create -n setuptools_bug python=3.10 setuptools -c conda-forge
$ conda activate setuptools_bug
$ mkdir mypackage
$ cat > mypackage/__init__.py
def hello():
  print('hello!')

$ cat > pyproject.toml 
[project]
name = "mypackage"
version = "0.0.1"
dependencies = [
    "requests",
    'importlib-metadata; python_version<"3.8"',
]

$ cat > repro.py
import setuptools
import pip
setuptools.setup(
    name='mypackage',
    version='0.0.1',
    packages=['mypackage'],
    install_requires=[
        'requests',
        'importlib-metadata; python_version == "3.8"',
    ],
)

$ python repro.py build
/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
  File "/net/cci-filer3/home/dwpaley/setuptools_bug/repro.py", line 3, in <module>
    setuptools.setup(
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
    _install_setup_requires(attrs)
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/setuptools/__init__.py", line 78, in _install_setup_requires
    dist.parse_config_files(ignore_option_errors=True)
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/setuptools/dist.py", line 848, in parse_config_files
    self._parse_config_files(filenames=inifiles)
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/setuptools/dist.py", line 690, in _parse_config_files
    filenames = self.find_config_files()
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 353, in find_config_files
    sys_dir = os.path.dirname(sys.modules['distutils'].__file__)
KeyError: 'distutils'

Second bug:

$ conda create -n setuptools_bug python=3.10 setuptools -c conda-forge
$ conda activate setuptools_bug
$ python
Python 3.10.4 | packaged by conda-forge | (main, Mar 24 2022, 17:39:04) [GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
>>> import setuptools
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 72, in do_override
    ensure_local_distutils()
  File "/net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 59, in ensure_local_distutils
    assert '_distutils' in core.__file__, core.__file__
AssertionError: /net/cci/dwpaley/miniconda3/envs/setuptools_bug/lib/python3.10/distutils/core.py

[abravalheri]

Hi @dwpaley, thanks for bringing this topic for discussion.

Some versions of pip curently don't seem to handle well situations when setuptools provides distutils (see pip's issue 8761) (this is a completely fair behaviour until distutils is officially abandoned in Python 3.12).

At the same time, setuptools needs to push its own bundled copy of distutils to be able to provide several improvements and fixes.

As a result, setuptools needs to pro-actively remove distutils from sys.modules every time pip is detected which will in turn trigger the behaviour that you are observing.

Unfortunately I don't think we can manage to combine usage of setuptools + pip for versions of Python < 3.10... Therefore this edge case would fall into the category (described in the CHANGELOG):

Setuptools overrides the stdlib distutils on import.
For environments or invocations where this behavior is undesirable,
users are provided with a temporary escape hatch. If the environment
variable SETUPTOOLS_USE_DISTUTILS is set to stdlib, Setuptools will
fall back to the legacy behavior. Use of this escape hatch is
discouraged, but it is provided to ease the transition while proper
fixes for edge cases can be addressed.

Or do you have a suggestion on how to implement a workaround? (If you do, please feel free to open a PR! We would be super happy to have a look).

[abravalheri]

It might also be the case this problem is not observed for Python >= 3.10

[dwpaley]

@abravalheri Thank you! I also observed this behavior (at least "bug 1") on Python 3.10. In any case we'll need to support older Python versions for a while. Thanks for the SETUPTOOLS_USE_DISTUTILS=stdlib trick. Maybe we can refactor out our pip and setuptools steps into separately dispatched processes, or else try setting the env variable. I don't currently have any workaround to suggest, but I'll reflect on it.
Sounds like this is already well known behavior so I have no objection if you'd like to close this.
Best,
Dan

[dwpaley]

Would you be open to a PR that adds warnings for incompatible combinations of imports? (maybe warn if importing pip after setuptools, and refuse more explicitly to import setuptools after pip?)

[abravalheri]

Hi @dwpaley, sorry for the delay.

Yes, that would be great. All PRs are welcome! Thank you very much.

(I am a bit curious though on how to do that without showing warnings during regular/standard pip operations...)

[pelson]

For the record (and to link the two issues), I added a "workaround" to this issue in #3439 (comment). I didn't validate that the resulting pip and setuptools are fully functional, but at least they continue to import.

[trevorboydsmith]

i was building a wheel file with the "manylinux2014_x86_64 that was recently build in 2023-08 and i ran across errors whenever i did (sanity check looking for pip,setuptools,wheel):

${python3_bin} -c "import pip,setuptools,wheel"

the issue is specific to python version >= 3.7

the fix i did was:

${python3_bin} -c "import pip"
${python3_bin} -c "import setuptools"
${python3_bin} -c "import wheel"