pip-25.0.1 runs setup.py with broken import behaviour #13289

julian-smith-artifex-com · 2025-03-19T15:12:54Z

Description

pip-25.0.1 appears to run setup.py in such a way that import platform picks up a local platform/ directory if it exists, instead of the built-in module.

This happens if platform/ is empty, or if it contains other files and directories without a __init__.py file.

I include a Python script that shows the broken behaviour.

Run with no args: ./pipbug.py.
If not run in a venv it reruns itself inside a new venv.
Then it creates a new project directory with a pyproject.toml file, a (non-working) setup.py and an empty platform/ directory.
Then it attempts to build a wheel with pip-24.3.1 and pip-25.0.1.

Expected behavior

import platform should get the built-in module regardless of whether there is a local platform directory.

pip version

25.0.1

Python version

3.11 and 3.12.

OS

Linux, OpenBSD.

How to Reproduce

Run this Python script as file pipbug.py:

#!/usr/bin/env python3

'''
pip-25.0.1 runs setup.py in such a way that `import platform` picks up local
`platform/` directory, not the built-in module.
'''

import os
import shlex
import shutil
import subprocess
import sys
import textwrap


filep = os.path.normpath(__file__)

def run(command, check=1):
    print(f'Running: {command}', flush=1)
    return subprocess.run(command, shell=1, check=check)

def main():
    # Create project directory {filep}_testdir
    testdir = f'{filep}_testdir'
    shutil.rmtree(testdir, ignore_errors=1)
    os.mkdir(testdir)
    # Create empty `platform/` directory.
    os.mkdir(f'{testdir}/platform')
    
    # Create (non-working) setup.py.
    with open(f'{testdir}/setup.py', 'w') as f:
        f.write(
                textwrap.dedent('''
                    import os
                    import platform
                    import sys
                    print(f'setup.py: {sys.path=}')
                    print(f'setup.py: {os.getcwd()=}')
                    print(f'setup.py: {dir(platform)=}')
                    print(f'setup.py: {platform.__file__=}')
                    print(f'setup.py: {getattr(platform, "__path__", None)=}')
                    print(f'setup.py: {platform.system()=}')
                    ''')
                )

    # Create pyproject.toml.
    with open(f'{testdir}/pyproject.toml', 'w') as f:
        f.write(
                textwrap.dedent('''
                    [build-system]
                    requires = []

                    # See pep-517.
                    #
                    build-backend = "setup"
                    backend-path = ["."]
                    ''')
                )

    # Attempt to create wheel with pip-24.3.1. This gives expected error:
    #   AttributeError: module 'setup' has no attribute 'build_wheel'
    print('=' * 80)
    print(f'Testing with pip-24.3.1')
    run(f'pip install pip==24.3.1')
    run(f'pip wheel -v {os.path.abspath(testdir)}', check=0)
    
    # Attempt to create wheel with pip-25.0.1. This gives different error:
    #   AttributeError: module 'platform' has no attribute 'system'
    print('=' * 80)
    print(f'Testing with pip-25.0.1')
    run(f'pip install pip==25.0.1')
    run(f'pip wheel -v {os.path.abspath(testdir)}', check=0)


if __name__ == '__main__':
    if sys.prefix == sys.base_prefix:
        # Not running in a venv. Rerun ourselves in a venv.
        command = f'{sys.executable} -m venv {filep}_venv'
        command += f' && . {filep}_venv/bin/activate'
        command += f' && python'
        for arg in sys.argv:
            command += f' {shlex.quote(arg)}'
        run(command)
    else:
        main()

Output

Example output on Linux is below.

With pip-24.3.1 we get expected error `AttributeError: module 'setup' has no attribute 'build_wheel'.
With pip-25.0.1 we get earlier error AttributeError: module 'platform' has no attribute 'system', because import platform has picked up the local empty platform directory instead of the built-in module platform.

> ./pipbug.py
Running: /usr/bin/python3 -m venv [...]/pipbug.py_venv && . [...]/pipbug.py_venv/bin/activate && python ./pipbug.py
================================================================================
Testing with pip-24.3.1
Running: pip install pip==24.3.1
Collecting pip==24.3.1
  Using cached pip-24.3.1-py3-none-any.whl.metadata (3.7 kB)
Using cached pip-24.3.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 25.0.1
    Uninstalling pip-25.0.1:
      Successfully uninstalled pip-25.0.1
Successfully installed pip-24.3.1

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
Running: pip wheel -v [...]/pipbug.py_testdir
Processing ./pipbug.py_testdir
  Getting requirements to build wheel: started
  Running command Getting requirements to build wheel
  setup.py: sys.path=['[...]/pipbug.py_testdir', '[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process', '/tmp/pip-build-env-4awji4kl/site', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/tmp/pip-build-env-4awji4kl/overlay/lib/python3.12/site-packages', '/tmp/pip-build-env-4awji4kl/normal/lib/python3.12/site-packages']
  setup.py: os.getcwd()='[...]/pipbug.py_testdir'
  setup.py: dir(platform)=['_Processor', '_WIN32_CLIENT_RELEASES', '_WIN32_SERVER_RELEASES', '__builtins__', '__cached__', '__copyright__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '__version__', '_comparable_version', '_default_architecture', '_follow_symlinks', '_get_machine_win32', '_java_getprop', '_mac_ver_xml', '_node', '_norm_version', '_os_release_cache', '_os_release_candidates', '_parse_os_release', '_platform', '_platform_cache', '_sys_version', '_sys_version_cache', '_syscmd_file', '_syscmd_ver', '_uname_cache', '_unknown_as_blank', '_ver_stages', '_win32_ver', '_wmi_query', 'architecture', 'collections', 'freedesktop_os_release', 'functools', 'itertools', 'java_ver', 'libc_ver', 'mac_ver', 'machine', 'node', 'os', 'platform', 'processor', 'python_branch', 'python_build', 'python_compiler', 'python_implementation', 'python_revision', 'python_version', 'python_version_tuple', 're', 'release', 'sys', 'system', 'system_alias', 'uname', 'uname_result', 'version', 'win32_edition', 'win32_is_iot', 'win32_ver']
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  setup.py: platform.__file__='/usr/lib/python3.12/platform.py'
  setup.py: getattr(platform, "__path__", None)=None
  setup.py: platform.system()='Linux'
  Running command Preparing metadata (pyproject.toml)
  setup.py: sys.path=['[...]/pipbug.py_testdir', '[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process', '/tmp/pip-build-env-4awji4kl/site', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/tmp/pip-build-env-4awji4kl/overlay/lib/python3.12/site-packages', '/tmp/pip-build-env-4awji4kl/normal/lib/python3.12/site-packages']
  setup.py: os.getcwd()='[...]/pipbug.py_testdir'
  setup.py: dir(platform)=['_Processor', '_WIN32_CLIENT_RELEASES', '_WIN32_SERVER_RELEASES', '__builtins__', '__cached__', '__copyright__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '__version__', '_comparable_version', '_default_architecture', '_follow_symlinks', '_get_machine_win32', '_java_getprop', '_mac_ver_xml', '_node', '_norm_version', '_os_release_cache', '_os_release_candidates', '_parse_os_release', '_platform', '_platform_cache', '_sys_version', '_sys_version_cache', '_syscmd_file', '_syscmd_ver', '_uname_cache', '_unknown_as_blank', '_ver_stages', '_win32_ver', '_wmi_query', 'architecture', 'collections', 'freedesktop_os_release', 'functools', 'itertools', 'java_ver', 'libc_ver', 'mac_ver', 'machine', 'node', 'os', 'platform', 'processor', 'python_branch', 'python_build', 'python_compiler', 'python_implementation', 'python_revision', 'python_version', 'python_version_tuple', 're', 'release', 'sys', 'system', 'system_alias', 'uname', 'uname_result', 'version', 'win32_edition', 'win32_is_iot', 'win32_ver']
  setup.py: platform.__file__='/usr/lib/python3.12/platform.py'
  setup.py: getattr(platform, "__path__", None)=None
  setup.py: platform.system()='Linux'
  Traceback (most recent call last):
    File "[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
      whl_basename = backend.build_wheel(metadata_directory, config_settings)
                     ^^^^^^^^^^^^^^^^^^^
  AttributeError: module 'setup' has no attribute 'build_wheel'
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  Ã� Preparing metadata (pyproject.toml) did not run successfully.
  â�� exit code: 1
  â�°â��> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: [...]/pipbug.py_venv/bin/python3 [...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpoiz5wz0s
  cwd: [...]/pipbug.py_testdir

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
error: metadata-generation-failed

Ã� Encountered error while generating package metadata.
â�°â��> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
================================================================================
Testing with pip-25.0.1
Running: pip install pip==25.0.1
Collecting pip==25.0.1
  Using cached pip-25.0.1-py3-none-any.whl.metadata (3.7 kB)
Using cached pip-25.0.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 24.3.1
    Uninstalling pip-24.3.1:
      Successfully uninstalled pip-24.3.1
Successfully installed pip-25.0.1
Running: pip wheel -v [...]/pipbug.py_testdir
Processing ./pipbug.py_testdir
  Getting requirements to build wheel: started
  Running command Getting requirements to build wheel
  setup.py: sys.path=['/tmp/pip-build-env-9swhkfon/site', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/tmp/pip-build-env-9swhkfon/overlay/lib/python3.12/site-packages', '/tmp/pip-build-env-9swhkfon/normal/lib/python3.12/site-packages']
  setup.py: os.getcwd()='[...]/pipbug.py_testdir'
  setup.py: dir(platform)=['__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
  setup.py: platform.__file__=None
  setup.py: getattr(platform, "__path__", None)=_NamespacePath(['[...]/pipbug.py_testdir/platform'])
  Traceback (most recent call last):
    File "[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
      main()
    File "[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
      json_out["return_val"] = hook(**hook_input["kwargs"])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 137, in get_requires_for_build_wheel
      backend = _build_backend()
                ^^^^^^^^^^^^^^^^
    File "[...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 70, in _build_backend
      obj = import_module(mod_path)
            ^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
    File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
    File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 999, in exec_module
    File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
    File "[...]/pipbug.py_testdir/setup.py", line 10, in <module>
      print(f'setup.py: {platform.system()=}')
                         ^^^^^^^^^^^^^^^
  AttributeError: module 'platform' has no attribute 'system'
  error: subprocess-exited-with-error
  
  Ã� Getting requirements to build wheel did not run successfully.
  â�� exit code: 1
  â�°â��> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: [...]/pipbug.py_venv/bin/python3 [...]/pipbug.py_venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp1lbpiwek
  cwd: [...]/pipbug.py_testdir
  Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error

Ã� Getting requirements to build wheel did not run successfully.
â�� exit code: 1
â�°â��> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Code of Conduct

I agree to follow the PSF Code of Conduct.

The text was updated successfully, but these errors were encountered:

julian-smith-artifex-com · 2025-03-19T15:17:06Z

The same problem also occurs on Windows (needs some minor modifications to the script).

notatallshaw · 2025-03-19T15:35:49Z

pip-25.0.1 appears to run setup.py in such a way that import platform picks up a local platform/ directory if it exists, instead of the built-in module.

Isn't this standard Python behavior?

If you create a directory or Python file in the same directory, then it becomes importable, and has higher precedence than standard library modules.

What else would you expect? What if someone was importing a local module with that name?

julian-smith-artifex-com · 2025-03-19T16:01:15Z

I don't think it is standard Python behaviour - i think a local directory needs to have a __init__.py file in order to be considered for import.

Without a __init__,py file, i think a local directory should not be considered for import. This is how plain Python behaves, and how setup.py behaves when invoked by older pip versions.

BTW i can't provide exact references to PEPs etc for this, import is pretty complicated and i don't pretend to understand all the details. But i do know that pip-25.0.1 has changed how setup's import statements behave.

pfmoore · 2025-03-19T16:19:22Z

I don't think it is standard Python behaviour - i think a local directory needs to have a __init__.py file in order to be considered for import.

No, see PEP 420 - Implicit Namespace Packages. Plain directories have been importable since Python 3.3.

RonnyPfannschmidt · 2025-03-19T16:24:49Z

could pypa/pyproject-hooks#199 be related

i suspect it may start to identify the local platform directory as viable namespace package with that but i haven't verified that idea

RonnyPfannschmidt · 2025-03-19T16:30:00Z

i suspect this may be a bug in behaviour of the meta path finder in pyproject hook

i believe its expected to try if another path hook find this as a non-namespace package before designating it as such

RonnyPfannschmidt · 2025-03-19T16:32:32Z

pypa/pyproject-hooks#207 looks like a very matching explanation

i beleive the reproducer can be turned into a regression test for pyproject hooks

julian-smith-artifex-com · 2025-03-19T22:43:00Z

I don't think it is standard Python behaviour - i think a local directory needs to have a __init__.py file in order to be considered for import.

No, see PEP 420 - Implicit Namespace Packages. Plain directories have been importable since Python 3.3.

Ah, apologies, i was completely unaware of this.

[But presumably there's still a problem with pip-25.0.1 because the built-in platform module should always be preferred over a bare platform/ directory?]

hroncok · 2025-03-20T19:26:25Z

About namespace packages: creating an empty platform directory in a directory where a script exists and executing the script with import platform in it imports the standard library platform module.

mjg · 2025-03-22T10:33:25Z

Is there something going on beyond messing with the path? If I put

#!/usr/bin/env python3

''' Work around pip 25/pyproject_hooks 1.2.0 path meddling '''

from platform import *

into platform/__init__.py (in the source tree, not sys) then nothing changes, whereas I would expect either

platform to have all attributes of platform (huh) or
python to complain about a circular import.

That is, unless python silently discards a circular import.

mjg · 2025-03-22T14:45:57Z

It gets even more interesting. Even if

I pop "platform" from sys.modules before the import
and sys.path does not contain the parent dir of the platform dir nor that dir

the import platform still identifies the dir platform as a namespace (when called via the setup hooks). So, maybe it's something that pip/pyproject_hooks does to sys.meta_path or importlib?

pip 25/pyproject_hooks 1.2.0 inserts `_BackendPathFinder` into `sys.meta_path` at index 0, and that makes the import machinery find the directory `platform` as a namespace instead of importing the standrd platform module. Work around by removing this finder from `sys.meta_path` if it is present at index 0. Upstream bug: pypa/pip#13289

mjg · 2025-03-22T16:23:55Z

So it seems to be pyproject_hooks injecting its own Finder into sys.meta_path. I don't know what the proper fix is in pip (not doing that, removing the Finder, fixing the Finder), but I know how to work around. PR for mupdf and FTBFS fix for Fedora package coming right up in the respective places.

notatallshaw · 2025-04-03T13:00:59Z

In pip 25.0 pyproject-hooks was upgraded from 1.0.0 to 1.2.0, so if it was a consequence of a change from pyproject-hooks then it should be one of these PRs: https://github.com/pypa/pyproject-hooks/pulls?q=is%3Apr+is%3Amerged+merged%3A2022-11-22..2024-09-29

To me, the obvious candidates are pypa/pyproject-hooks#165 and the follow up PR pypa/pyproject-hooks#193

mjg · 2025-04-03T13:24:53Z

AFAICT it is commit 084b02e in pyprojetc-hooks which looks like a big squash of several commits. It is beyond me why people do that - it prevents isolated reverts, let alone bisecting.
In pip, you do not use submodules but import a source dump of vendored modules (I know why) in a commit which contains other adjustments also. Again ...
This makes it difficult for me to work on and suggest a change, blame it on my incompatibility with this kind of git usage, sorry. All I can tell you is that on the pip consumer side, I work around like this:

ArtifexSoftware/mupdf@8c43f4f

It's the last commit in a PR which - originally - consisted of 1 commit only but got rebased/rewritten meanwhile, it seems. So it clearly is the _BackendPathFinder in the changed sys.meta_path which inserts those namespaces. Cleaning up sys.meta_path helps, as would fixing the finder (I don't know how, I've tried).

pip 25/pyproject_hooks 1.2.0 inserts `_BackendPathFinder` into `sys.meta_path` at index 0, and that makes the import machinery find the directory `platform` as a namespace instead of importing the standrd platform module. Work around by removing this finder from `sys.meta_path` if it is present at index 0. Upstream bug: pypa/pip#13289

julian-smith-artifex-com added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Mar 19, 2025

notatallshaw added project: vendored dependency Related to a vendored dependency and removed S: needs triage Issues/PRs that need to be triaged labels Mar 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pip-25.0.1 runs setup.py with broken import behaviour #13289

pip-25.0.1 runs setup.py with broken import behaviour #13289

julian-smith-artifex-com commented Mar 19, 2025 •

edited

Loading

julian-smith-artifex-com commented Mar 19, 2025

notatallshaw commented Mar 19, 2025

julian-smith-artifex-com commented Mar 19, 2025

pfmoore commented Mar 19, 2025

RonnyPfannschmidt commented Mar 19, 2025

RonnyPfannschmidt commented Mar 19, 2025

RonnyPfannschmidt commented Mar 19, 2025

julian-smith-artifex-com commented Mar 19, 2025

hroncok commented Mar 20, 2025

mjg commented Mar 22, 2025

mjg commented Mar 22, 2025 •

edited

Loading

mjg commented Mar 22, 2025

notatallshaw commented Apr 3, 2025

mjg commented Apr 3, 2025

pip-25.0.1 runs setup.py with broken import behaviour #13289

pip-25.0.1 runs setup.py with broken import behaviour #13289

Comments

julian-smith-artifex-com commented Mar 19, 2025 • edited Loading

Description

Expected behavior

pip version

Python version

OS

How to Reproduce

Output

Code of Conduct

julian-smith-artifex-com commented Mar 19, 2025

notatallshaw commented Mar 19, 2025

julian-smith-artifex-com commented Mar 19, 2025

pfmoore commented Mar 19, 2025

RonnyPfannschmidt commented Mar 19, 2025

RonnyPfannschmidt commented Mar 19, 2025

RonnyPfannschmidt commented Mar 19, 2025

julian-smith-artifex-com commented Mar 19, 2025

hroncok commented Mar 20, 2025

mjg commented Mar 22, 2025

mjg commented Mar 22, 2025 • edited Loading

mjg commented Mar 22, 2025

notatallshaw commented Apr 3, 2025

mjg commented Apr 3, 2025

julian-smith-artifex-com commented Mar 19, 2025 •

edited

Loading

mjg commented Mar 22, 2025 •

edited

Loading