`_distutils_hack` fails if called anything but `get-pip.py` #3439

balupton · 2022-01-19T15:48:56Z

Description

Pip fails to install:

which python
# /usr/local/opt/python/libexec/bin/python
/usr/local/opt/python/libexec/bin/python --version
# Python 3.9.10
down https://bootstrap.pypa.io/get-pip.py --destination=/var/folders/wm/8g56ry4s47z_hbt0tgv0m5gr0000gn/T/tmp.vv6opZCN
# down is an internal command
python /var/folders/wm/8g56ry4s47z_hbt0tgv0m5gr0000gn/T/tmp.vv6opZCN --user
# fails and outputs the next code block

/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py:17: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is in/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py:30File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/overrideFile "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py:17: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
  File "/var/folders/wm/8g56ry4s47z_hbt0tgv0m5gr0000gn/T/tmp.vv6opZCN", line 27081, in <module>
    main()
  File "/var/folders/wm/8g56ry4s47z_hbt0tgv0m5gr0000gn/T/tmp.vv6opZCN", line 139, in main
    bootstrap(tmpdir=tmpdir)
  File "/var/folders/wm/8g56ry4s47z_hbt0tgv0m5gr0000gn/T/tmp.vv6opZCN", line 120, in bootstrap
    args = determine_pip_install_arguments()
  File "/var/folders/wm/8g56ry4s47z_hbt0tgv0m5gr0000gn/T/tmp.vv6opZCN", line 65, in determine_pip_install_arguments
    import setuptools  # noqa
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 71, in do_override
    ensure_local_distutils()
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 59, in ensure_local_distutils
    assert '_distutils' in core.__file__, core.__file__
AssertionError: /usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py

Expected behavior

it to work fine

pip version

latest

Python version

3.9.10

OS

21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:54 PST 2021; root:xnu-8019.61.5~1/RELEASE_X86_64 x86_64

How to Reproduce

this works fine:

brew install python
wget https://bootstrap.pypa.io/get-pip.py
chmod +x ./get-pip.py
python ./get-pip.py --user

this fails, which is common when say saving the installer to a randomly generated temp file:

brew install python
wget https://bootstrap.pypa.io/get-pip.py
mv ./get-pip.py ./any-other-filename.py
chmod +x ./any-other-filename.py
python ./any-other-filename.py --user

Output

Warning: [email protected] 3.9.10 is already installed and up-to-date.
To reinstall 3.9.10, run:
  brew reinstall [email protected]
--2022-01-19 23:48:26--  https://bootstrap.pypa.io/get-pip.py
Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 151.101.52.175, 2a04:4e42:d::175
Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|151.101.52.175|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2159352 (2.1M) [text/x-python]
Saving to: ‘get-pip.py’

get-pip.py                        100%[============================================================>]   2.06M  1.65MB/s    in 1.2s    

2022-01-19 23:48:28 (1.65 MB/s) - ‘get-pip.py’ saved [2159352/2159352]

/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py:17: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
  File "/Users/balupton/./any-other-filename.py", line 27081, in <module>
    main()
  File "/Users/balupton/./any-other-filename.py", line 139, in main
    bootstrap(tmpdir=tmpdir)
  File "/Users/balupton/./any-other-filename.py", line 120, in bootstrap
    args = determine_pip_install_arguments()
  File "/Users/balupton/./any-other-filename.py", line 65, in determine_pip_install_arguments
    import setuptools  # noqa
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 71, in do_override
    ensure_local_distutils()
  File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 59, in ensure_local_distutils
    assert '_distutils' in core.__file__, core.__file__
AssertionError: /usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py

Code of Conduct

I agree to follow the PSF Code of Conduct.

The text was updated successfully, but these errors were encountered:

- `setup-python` fix https://github.com/pypa/pip/issues/10809 - `select-hosts` fix on macos due to /etc/hosts permission issue - `setup-mac-brew` fix go permission error on clean - `fs-structure` do not output filenames, so `is-same` works on files - `chmod-helper` integrated into `fs-own` via `--permissions=...` and incorporated flags - `fs-own` now uses correct admin user and group on macos via `--admin` - `echo-eval` now outputs quoted arguments - `shims.bash` now includes `quote` shim for `@Q` - usage of this should improve bash v3 compat in more places - `fs-temp` now includes `--filename` and `--xdg` args, and removes directory argument for only `--directory=...` option - updated users of this script to make use of new flags - fix styling in `is-same`, `macos-(drive|state|theme)`

pradyunsg · 2022-01-19T16:13:30Z

Pinging @jaraco, since this comes out of the distutils hack that setuptools has.

pradyunsg · 2022-07-09T00:55:51Z

This is realistically an issue with setuptools' hack, so I've moved the issue over.

jaraco · 2022-07-10T14:14:51Z

I'm sorry I missed the ping. I have hundreds of unread messages in my email inbox, so would have gotten to it eventually (most likely).

The distutils hack does have an exemption for pip to force it to use the stdlib distutils, but nothing specific to "get-pip.py", so I don't yet understand why the name of get-pip is relevant.

jaraco · 2022-07-10T14:25:52Z

Attempting to replicate the issue, I created this Dockerfile:

FROM ubuntu:jammy
RUN apt update
RUN apt upgrade -y
RUN apt install -y python3 python3-distutils wget
RUN wget https://bootstrap.pypa.io/get-pip.py -O othername.py
CMD python3 othername.py --user

Unfortunately, it works fine:

 draft $ docker run -it @$(docker build -q .)
Collecting pip
  Downloading pip-22.1.2-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 10.8 MB/s eta 0:00:00
Collecting setuptools
  Downloading setuptools-63.1.0-py3-none-any.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 16.5 MB/s eta 0:00:00
Collecting wheel
  Downloading wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools, pip
  WARNING: The script wheel is installed in '/root/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts pip, pip3 and pip3.10 are installed in '/root/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.1.2 setuptools-63.1.0 wheel-0.37.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Therefore, the issue appears to be restricted to some other factor in the user's environment (Python version, pip version, OS, Homebrew patches, ...).

Is it possible someone could create a Dockerfile that can replicate the issue?

jaraco · 2022-10-04T14:09:29Z

I've moved your comment to a new issue because it's a different case.

jaraco · 2022-10-04T14:20:14Z

I thought maybe my previous attempt to reproduce the issue was because of the lack of setuptools, so I tried this approach also, but unsuccessfully:

FROM ubuntu:jammy
RUN apt update
RUN apt upgrade -y
RUN apt install -y python3 python3-distutils wget
RUN wget https://bootstrap.pypa.io/get-pip.py -O othername.py
RUN python3 othername.py --user
RUN python3 -m pip install setuptools
RUN python3 -m pip uninstall -y pip
CMD python3 othername.py --user

balupton · 2022-10-04T14:31:29Z

If I've been the only person who has experienced this, you can happily ignore the issue, as my script implemented the naming solution which worked around the issue.

jaraco · 2022-10-04T14:32:58Z

I see now that in pypa/get-pip#140, the get-pip script was updated not to import setuptools, and that script was probably updated in get-pip.py in late January, so I suspect the root cause of this issue has been addressed.

If any one else encounters this issue and can reproduce it (preferably in a Docker image) using the latest get-pip, please let us know.

sharpicx · 2023-06-03T03:35:24Z

im still gettin an error on debian, i cannot see any correction here.

edit: after installing setuptools = 49.6. now im getting a new error

edit: by using this link https://bootstrap.pypa.io/pip/2.7/get-pip.py, i finally have pip2 installed.

balupton changed the title ~~pip fails to install if the installer is called anything but get-pip.py~~ installer fails if called anything butget-pip.py Jan 19, 2022

balupton changed the title ~~installer fails if called anything butget-pip.py~~ installer fails if called anything but get-pip.py Jan 19, 2022

pradyunsg changed the title ~~installer fails if called anything but get-pip.py~~ _distutils_hack fails if called anything but get-pip.py Jan 19, 2022

vyasr mentioned this issue May 24, 2022

Issue template versions command no longer works pandas-dev/pandas#47106

Closed

This comment was marked as off-topic.

Sign in to view

grv87 mentioned this issue Jul 8, 2022

AssertionError no _distutils when running get-pip with Setuptools and USE_DISTUTILS=local #2993

Closed

pradyunsg transferred this issue from pypa/pip Jul 9, 2022

jaraco self-assigned this Jul 10, 2022

jaraco removed their assignment Jul 10, 2022

This comment was marked as off-topic.

Sign in to view

pelson mentioned this issue Oct 4, 2022

[BUG] Importing pip breaks setuptools #3297

Open

jaraco mentioned this issue Oct 4, 2022

AssertionError when importing pip before setuptools #3621

Open

jaraco closed this as completed Oct 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`_distutils_hack` fails if called anything but `get-pip.py` #3439

`_distutils_hack` fails if called anything but `get-pip.py` #3439

balupton commented Jan 19, 2022 •

edited

Loading

pradyunsg commented Jan 19, 2022 •

edited

Loading

This comment was marked as off-topic.

pradyunsg commented Jul 9, 2022

jaraco commented Jul 10, 2022

jaraco commented Jul 10, 2022 •

edited

Loading

This comment was marked as off-topic.

jaraco commented Oct 4, 2022

jaraco commented Oct 4, 2022

balupton commented Oct 4, 2022 •

edited

Loading

jaraco commented Oct 4, 2022

sharpicx commented Jun 3, 2023 •

edited

Loading

_distutils_hack fails if called anything but get-pip.py #3439

_distutils_hack fails if called anything but get-pip.py #3439

Comments

balupton commented Jan 19, 2022 • edited Loading

Description

Expected behavior

pip version

Python version

OS

How to Reproduce

Output

Code of Conduct

pradyunsg commented Jan 19, 2022 • edited Loading

This comment was marked as off-topic.

pradyunsg commented Jul 9, 2022

jaraco commented Jul 10, 2022

jaraco commented Jul 10, 2022 • edited Loading

This comment was marked as off-topic.

jaraco commented Oct 4, 2022

jaraco commented Oct 4, 2022

balupton commented Oct 4, 2022 • edited Loading

jaraco commented Oct 4, 2022

sharpicx commented Jun 3, 2023 • edited Loading

`_distutils_hack` fails if called anything but `get-pip.py` #3439

`_distutils_hack` fails if called anything but `get-pip.py` #3439

balupton commented Jan 19, 2022 •

edited

Loading

pradyunsg commented Jan 19, 2022 •

edited

Loading

jaraco commented Jul 10, 2022 •

edited

Loading

balupton commented Oct 4, 2022 •

edited

Loading

sharpicx commented Jun 3, 2023 •

edited

Loading