diff --git a/docs/html/cli/pip.rst b/docs/html/cli/pip.rst index d5b6b034c80..77aac009fa1 100644 --- a/docs/html/cli/pip.rst +++ b/docs/html/cli/pip.rst @@ -74,178 +74,11 @@ when decision is needed. *(a)abort* Abort pip and return non-zero exit status. -.. _`build-interface`: - Build System Interface ====================== -pip builds packages by invoking the build system. By default, builds will use -``setuptools``, but if a project specifies a different build system using a -``pyproject.toml`` file, as per :pep:`517`, pip will use that instead. As well -as package building, the build system is also invoked to install packages -direct from source. This is handled by invoking the build system to build a -wheel, and then installing from that wheel. The built wheel is cached locally -by pip to avoid repeated identical builds. - -The current interface to the build system is via the ``setup.py`` command line -script - all build actions are defined in terms of the specific ``setup.py`` -command line that will be run to invoke the required action. - -Setuptools Injection -~~~~~~~~~~~~~~~~~~~~ - -When :pep:`517` is not used, the supported build system is ``setuptools``. -However, not all packages use ``setuptools`` in their build scripts. To support -projects that use "pure ``distutils``", pip injects ``setuptools`` into -``sys.modules`` before invoking ``setup.py``. The injection should be -transparent to ``distutils``-based projects, but 3rd party build tools wishing -to provide a ``setup.py`` emulating the commands pip requires may need to be -aware that it takes place. - -Projects using :pep:`517` *must* explicitly use setuptools - pip does not do -the above injection process in this case. - -Build System Output -~~~~~~~~~~~~~~~~~~~ - -Any output produced by the build system will be read by pip (for display to the -user if requested). In order to correctly read the build system output, pip -requires that the output is written in a well-defined encoding, specifically -the encoding the user has configured for text output (which can be obtained in -Python using ``locale.getpreferredencoding``). If the configured encoding is -ASCII, pip assumes UTF-8 (to account for the behaviour of some Unix systems). - -Build systems should ensure that any tools they invoke (compilers, etc) produce -output in the correct encoding. In practice - and in particular on Windows, -where tools are inconsistent in their use of the "OEM" and "ANSI" codepages - -this may not always be possible. pip will therefore attempt to recover cleanly -if presented with incorrectly encoded build tool output, by translating -unexpected byte sequences to Python-style hexadecimal escape sequences -(``"\x80\xff"``, etc). However, it is still possible for output to be displayed -using an incorrect encoding (mojibake). - -Under :pep:`517`, handling of build tool output is the backend's responsibility, -and pip simply displays the output produced by the backend. (Backends, however, -will likely still have to address the issues described above). - -PEP 517 and 518 Support -~~~~~~~~~~~~~~~~~~~~~~~ - -As of version 10.0, pip supports projects declaring dependencies that are -required at install time using a ``pyproject.toml`` file, in the form described -in :pep:`518`. When building a project, pip will install the required -dependencies locally, and make them available to the build process. -Furthermore, from version 19.0 onwards, pip supports projects specifying the -build backend they use in ``pyproject.toml``, in the form described in -:pep:`517`. - -When making build requirements available, pip does so in an *isolated -environment*. That is, pip does not install those requirements into the user's -``site-packages``, but rather installs them in a temporary directory which it -adds to the user's ``sys.path`` for the duration of the build. This ensures -that build requirements are handled independently of the user's runtime -environment. For example, a project that needs a recent version of setuptools -to build can still be installed, even if the user has an older version -installed (and without silently replacing that version). - -In certain cases, projects (or redistributors) may have workflows that -explicitly manage the build environment. For such workflows, build isolation -can be problematic. If this is the case, pip provides a -``--no-build-isolation`` flag to disable build isolation. Users supplying this -flag are responsible for ensuring the build environment is managed -appropriately (including ensuring that all required build dependencies are -installed). - -By default, pip will continue to use the legacy (direct ``setup.py`` execution -based) build processing for projects that do not have a ``pyproject.toml`` file. -Projects with a ``pyproject.toml`` file will use a :pep:`517` backend. Projects -with a ``pyproject.toml`` file, but which don't have a ``build-system`` section, -will be assumed to have the following backend settings:: - - [build-system] - requires = ["setuptools>=40.8.0", "wheel"] - build-backend = "setuptools.build_meta:__legacy__" - -.. note:: - - ``setuptools`` 40.8.0 is the first version of setuptools that offers a - :pep:`517` backend that closely mimics directly executing ``setup.py``. - -If a project has ``[build-system]``, but no ``build-backend``, pip will also use -``setuptools.build_meta:__legacy__``, but will expect the project requirements -to include ``setuptools`` and ``wheel`` (and will report an error if the -installed version of ``setuptools`` is not recent enough). - -If a user wants to explicitly request :pep:`517` handling even though a project -doesn't have a ``pyproject.toml`` file, this can be done using the -``--use-pep517`` command line option. Similarly, to request legacy processing -even though ``pyproject.toml`` is present, the ``--no-use-pep517`` option is -available (although obviously it is an error to choose ``--no-use-pep517`` if -the project has no ``setup.py``, or explicitly requests a build backend). As -with other command line flags, pip recognises the ``PIP_USE_PEP517`` -environment veriable and a ``use-pep517`` config file option (set to true or -false) to set this option globally. Note that overriding pip's choice of -whether to use :pep:`517` processing in this way does *not* affect whether pip -will use an isolated build environment (which is controlled via -``--no-build-isolation`` as noted above). - -Except in the case noted above (projects with no :pep:`518` ``[build-system]`` -section in ``pyproject.toml``), pip will never implicitly install a build -system. Projects **must** ensure that the correct build system is listed in -their ``requires`` list (this applies even if pip assumes that the -``setuptools`` backend is being used, as noted above). - -.. _pep-518-limitations: - -**Historical Limitations**: - -* ``pip<18.0``: only supports installing build requirements from wheels, and - does not support the use of environment markers and extras (only version - specifiers are respected). - -* ``pip<18.1``: build dependencies using .pth files are not properly supported; - as a result namespace packages do not work under Python 3.2 and earlier. - -Future Developments -~~~~~~~~~~~~~~~~~~~ - -:pep:`426` notes that the intention is to add hooks to project metadata in -version 2.1 of the metadata spec, to explicitly define how to build a project -from its source. Once this version of the metadata spec is final, pip will -migrate to using that interface. At that point, the ``setup.py`` interface -documented here will be retained solely for legacy purposes, until projects -have migrated. - -Specifically, applications should *not* expect to rely on there being any form -of backward compatibility guarantees around the ``setup.py`` interface. - - -Build Options -~~~~~~~~~~~~~ - -The ``--global-option`` and ``--build-option`` arguments to the ``pip install`` -and ``pip wheel`` inject additional arguments into the ``setup.py`` command -(``--build-option`` is only available in ``pip wheel``). These arguments are -included in the command as follows: - -.. tab:: Unix/macOS - - .. code-block:: console - - python setup.py BUILD COMMAND - -.. tab:: Windows - - .. code-block:: shell - - py setup.py BUILD COMMAND - -The options are passed unmodified, and presently offer direct access to the -distutils command line. Use of ``--global-option`` and ``--build-option`` -should be considered as build system dependent, and may not be supported in the -current form if support for alternative build systems is added to pip. - +This is now covered in :doc:`../reference/build-system/index`. .. _`General Options`: diff --git a/docs/html/cli/pip_install.rst b/docs/html/cli/pip_install.rst index 6b4d3b1a2ba..d9af1b1e8bd 100644 --- a/docs/html/cli/pip_install.rst +++ b/docs/html/cli/pip_install.rst @@ -419,7 +419,7 @@ with other repeatability strategies. (rare) packages that use it will cause those dependencies to be downloaded by setuptools directly, skipping pip's hash-checking. If you need to use such a package, see :ref:`Controlling - setup_requires`. + setup_requires `. .. warning:: @@ -537,83 +537,10 @@ the project path. This is one advantage over just using ``setup.py develop``, which creates the "egg-info" directly relative the current working directory. -.. _`controlling-setup-requires`: - -Controlling setup_requires --------------------------- - -Setuptools offers the ``setup_requires`` `setup() keyword -`_ -for specifying dependencies that need to be present in order for the -``setup.py`` script to run. Internally, Setuptools uses ``easy_install`` -to fulfill these dependencies. - -pip has no way to control how these dependencies are located. None of the -package index options have an effect. - -The solution is to configure a "system" or "personal" `Distutils configuration -file -`_ to -manage the fulfillment. - -For example, to have the dependency located at an alternate index, add this: - -:: - - [easy_install] - index_url = https://my.index-mirror.com - -To have the dependency located from a local directory and not crawl PyPI, add this: - -:: - - [easy_install] - allow_hosts = '' - find_links = file:///path/to/local/archives/ - - Build System Interface ---------------------- -In order for pip to install a package from source, ``setup.py`` must implement -the following commands:: - - setup.py egg_info [--egg-base XXX] - setup.py install --record XXX [--single-version-externally-managed] [--root XXX] [--compile|--no-compile] [--install-headers XXX] - -The ``egg_info`` command should create egg metadata for the package, as -described in the setuptools documentation at -https://setuptools.readthedocs.io/en/latest/userguide/commands.html#egg-info-create-egg-metadata-and-set-build-tags - -The ``install`` command should implement the complete process of installing the -package to the target directory XXX. - -To install a package in "editable" mode (``pip install -e``), ``setup.py`` must -implement the following command:: - - setup.py develop --no-deps - -This should implement the complete process of installing the package in -"editable" mode. - -All packages will be attempted to built into wheels:: - - setup.py bdist_wheel -d XXX - -One further ``setup.py`` command is invoked by ``pip install``:: - - setup.py clean - -This command is invoked to clean up temporary commands from the build. (TODO: -Investigate in more detail when this command is required). - -No other build system commands are invoked by the ``pip install`` command. - -Installing a package from a wheel does not invoke the build system at all. - -.. _PyPI: https://pypi.org/ -.. _setuptools extras: https://setuptools.readthedocs.io/en/latest/userguide/dependency_management.html#optional-dependencies - +This is now covered in :doc:`../reference/build-system/index`. .. _`pip install Options`: @@ -906,3 +833,6 @@ Examples .. [1] This is true with the exception that pip v7.0 and v7.0.1 required quotes around specifiers containing environment markers in requirement files. + +.. _setuptools extras: https://setuptools.readthedocs.io/en/latest/userguide/dependency_management.html#optional-dependencies +.. _PyPI: https://pypi.org/ diff --git a/docs/html/cli/pip_wheel.rst b/docs/html/cli/pip_wheel.rst index c2a9543fc99..bf371f28506 100644 --- a/docs/html/cli/pip_wheel.rst +++ b/docs/html/cli/pip_wheel.rst @@ -28,60 +28,7 @@ Description Build System Interface ---------------------- -In order for pip to build a wheel, ``setup.py`` must implement the -``bdist_wheel`` command with the following syntax: - -.. tab:: Unix/macOS - - .. code-block:: shell - - python setup.py bdist_wheel -d TARGET - -.. tab:: Windows - - .. code-block:: shell - - py setup.py bdist_wheel -d TARGET - - -This command must create a wheel compatible with the invoking Python -interpreter, and save that wheel in the directory TARGET. - -No other build system commands are invoked by the ``pip wheel`` command. - -Customising the build -^^^^^^^^^^^^^^^^^^^^^ - -It is possible using ``--global-option`` to include additional build commands -with their arguments in the ``setup.py`` command. This is currently the only -way to influence the building of C extensions from the command line. For -example: - -.. tab:: Unix/macOS - - .. code-block:: shell - - python -m pip wheel --global-option bdist_ext --global-option -DFOO wheel - -.. tab:: Windows - - .. code-block:: shell - - py -m pip wheel --global-option bdist_ext --global-option -DFOO wheel - - -will result in a build command of - -:: - - setup.py bdist_ext -DFOO bdist_wheel -d TARGET - -which passes a preprocessor symbol to the extension build. - -Such usage is considered highly build-system specific and more an accident of -the current implementation than a supported interface. - - +This is now covered in :doc:`../reference/build-system/index`. Options ======= diff --git a/docs/html/reference/build-system/index.md b/docs/html/reference/build-system/index.md new file mode 100644 index 00000000000..ed43fec3778 --- /dev/null +++ b/docs/html/reference/build-system/index.md @@ -0,0 +1,127 @@ +(build-interface)= + +# Build System Interface + +When dealing with installable source distributions of a package, pip does not +directly handle the build process for the package. This responsibility is +delegated to "build backends" -- also known as "build systems". This means +that pip needs an interface, to interact with these build backends. + +There are two main interfaces that pip uses for these interactions: + +```{toctree} +:hidden: + +pyproject-toml +setup-py +``` + + +[`pyproject.toml` based](pyproject-toml) +: Standards-backed interface, that has explicit declaration and management of + build dependencies. + +[`setup.py` based](setup-py) +: Legacy interface, that we're working to migrate users away from. Has no good + mechanisms to declare build dependencies. + + +Details on the individual interfaces can be found on their dedicated pages, +linked above. This document covers the nuances around which build system +interface pip will use for a project, as well as details that apply to all +the build system interfaces that pip may use. + +## Determining which build system interface is used + +Currently, pip uses the `pyproject.toml` based build system interface, if a +`pyproject.toml` file exists. If not, the legacy build system interface is used. +The intention is to switch to using the `pyproject.toml` build system interface +unconditionally and to drop support for the legacy build system interface at +some point in the future. + +When performing a build, pip will mention which build system interface it is +using. Typically, this will take the form of a message like: + +```none +Building wheel for pip (pyproject.toml)... done +``` + +```none +Building wheel for pip (setup.py)... done +``` + +The content in the brackets, refers to which build system interface is being +used. + +```{versionchanged} 21.3 +The output uses "pyproject.toml" instead of "PEP 517" to refer to be +`pyproject.toml` based build system interface. +``` + +## Controlling which build system interface is used + +The [`--use-pep517`](install_--use-pep517) flag (and corresponding environment +variable: `PIP_USE_PEP517`) can be used to force all packages to build using +the `pyproject.toml` based build system interface. There is no way to force +the use of the legacy build system interface. + +(controlling-setup_requires)= + +## Controlling `setup_requires` + +```{hint} +This is only relevant for projects that use setuptools as the build backend, +and use the `setup_requires` keyword argument in their setup.py file. +``` + +The `setup_requires` argument in `setup.py` is used to specify build-time +dependencies for a package. This has been superseded by the +`build-system.requires` key in `pyproject.toml` files (per {pep}`518`). +However, there are situations where you might encounter a package that uses +`setup_requires` (eg: the package has not been updated to use the newer +approach yet!). + +If you control the package, consider adding a `pyproject.toml` file to utilise +the modern build system interface. That avoids invoking the problematic +behaviour by deferring to pip for the installations. + +For the end users, the best solution for dealing with packages with +`setup_requires` is to install the packages listed in `setup_requires` +beforehand, using a prior `pip install` command. This is because there is no +way to control how these dependencies are located by `easy_install`, or how +setuptools will invoke `pip` using pip's command line options -- which makes it +tricky to get things working appropriately. + +If you wish to ensure that `easy_install` invocations do not reach out to PyPI, +you will need to configure its behaviour using a +[`distutils` configuration file][distutils-config]. Here are some examples: + +- To have the dependency located at an alternate index with `easy_install` + + ```ini + [easy_install] + index_url = https://my.index-mirror.com + ``` + +- To have the dependency located from a local directory and not crawl PyPI, add this: + + ```ini + [easy_install] + allow_hosts = '' + find_links = file:///path/to/local/archives/ + ``` + +```{admonition} Historical context +`setuptools < 52.0` will use `easy_install` to try to fulfill `setup_requires` +dependencies, which can result in weird failures -- `easy_install` does not +understand many of the modern Python packaging standards, and will usually +attempt to install incompatible package versions or to build packages +incorrectly. It also generates improper script wrappers, which don't do the +right thing in many situations. + +Newer versions of `setuptools` will use `pip` for these installations, but have +limited ability to pass through any command line arguments. This can also result +in weird failures and subtly-incorrect behaviour. +``` + +[distutils-config]: https://docs.python.org/3/install/index.html#distutils-configuration-files diff --git a/docs/html/reference/build-system/pyproject-toml.md b/docs/html/reference/build-system/pyproject-toml.md new file mode 100644 index 00000000000..ee93df034a7 --- /dev/null +++ b/docs/html/reference/build-system/pyproject-toml.md @@ -0,0 +1,146 @@ +# `pyproject.toml` + +```{versionadded} 10.0 + +``` + +Modern Python packages can contain a `pyproject.toml` file, first introduced in +{pep}`518` and later expanded in {pep}`517`, {pep}`621` and {pep}`660`. +This file contains build system requirements and information, which are used by +pip to build the package. + +## Build process + +The overall process for building a package is: + +- Create an isolated build environment. +- Populate the build environment with build dependencies. +- Generate the package's metadata, if necessary and possible. +- Generate a wheel for the package. + +The wheel can then be used to perform an installation, if necessary. + +### Build Isolation + +For building packages using this interface, pip uses an _isolated environment_. +That is, pip will install build-time Python dependencies in a temporary +directory which will be added to `sys.path` for the build commands. This ensures +that build requirements are handled independently of the user's runtime +environment. + +For example, a project that needs an older version of setuptools to build can +still be installed, even if the user has an newer version installed (and +without silently replacing that version). + +### Build-time dependencies + +Introduced in {pep}`518`, the `build-system.requires` key in the +`pyproject.toml` file is a list of requirement specifiers for build-time +dependencies of a package. + +```toml +[build-system] +requires = ["setuptools ~= 58.0", "cython ~= 0.29.0"] +``` + +It is also possible for a build backend to provide dynamically calculated +build dependencies, using {pep}`517`'s `get_requires_for_build_wheel` hook. This +hook will be called by pip, and dependencies it describes will also be installed +in the build environment. For example, newer versions of setuptools expose the +contents of `setup_requires` to pip via this hook. + +### Metadata Generation + +```{versionadded} 19.0 + +``` + +Once the build environment has been created and populated with build-time +dependencies, `pip` will usually need metadata about a package (name, version, +dependencies, and more). + +If {pep}`517`'s `prepare_metadata_for_build_wheel` hook is provided by the +build backend, that will be used to generate the packages' metadata. Otherwise, +a wheel will be generated (as described below) and the metadata contained +within such a wheel will be used. + +### Wheel Generation + +```{versionadded} 19.0 + +``` + +For generating a wheel, pip uses the {pep}`517` `build_wheel` hook that has +to be provided by the build backend. The build backend will generate a wheel, +which may involve compiling extension code written in C/C++ (or other +languages). + +Wheels generated using this mechanism can be [cached](wheel-caching) for reuse, +to speed up future installations. + +### Editable Installation + +```{versionadded} 21.3 + +``` + +For performing editable installs, pip will use {pep}`660` +`build_wheel_for_editable` hook that has to be provided by the build backend. +The wheels generated using this mechanism are not cached. + +```{admonition} Compatibility fallback +If this hook is missing on the build backend _and_ there's a `setup.py` file +in the project, pip will fallback to the legacy setup.py-based editable +installation. + +This is considered a stopgap solution until setuptools adds support for +{pep}`660`, at which point this functionality will be removed; following pip's +regular {ref}`deprecation policy `. +``` + +## Build output + +It is the responsibility of the build backend to ensure that the output is +in the correct encoding, as described in {pep}`517`. This likely involves +dealing with [the same challenges as pip has for legacy builds](build-output). + +## Fallback Behaviour + +If a project does not have a `pyproject.toml` file containing a `build-system` +section, it will be assumed to have the following backend settings: + +```toml +[build-system] +requires = ["setuptools>=40.8.0", "wheel"] +build-backend = "setuptools.build_meta:__legacy__" +``` + +If a project has a `build-system` section but no `build-backend`, then: + +- It is expected to include `setuptools` and `wheel` as build requirements. An + error is reported if the available version of `setuptools` is not recent + enough. + +- The `setuptools.build_meta:__legacy__` build backend will be used. + +## Disabling build isolation + +This can be disabled using the `--no-build-isolation` flag -- users supplying +this flag are responsible for ensuring the build environment is managed +appropriately, including ensuring that all required build-time dependencies are +installed, since pip does not manage build-time dependencies when this flag is +passed. + +## Historical notes + +As this feature was incrementally rolled out, there have been various notable +changes and improvements in it. + +- setuptools 40.8.0 is the first version of setuptools that offers a + {pep}`517` backend that closely mimics directly executing `setup.py`. +- Prior to pip 18.0, pip only supports installing build requirements from + wheels, and does not support the use of environment markers and extras (only + version specifiers are respected). +- Prior to pip 18.1, build dependencies using `.pth` files are not properly + supported; as a result namespace packages do not work under Python 3.2 and + earlier. diff --git a/docs/html/reference/build-system/setup-py.md b/docs/html/reference/build-system/setup-py.md new file mode 100644 index 00000000000..722a05fa9ae --- /dev/null +++ b/docs/html/reference/build-system/setup-py.md @@ -0,0 +1,136 @@ +# `setup.py` (legacy) + +Prior to the introduction of pyproject.toml-based builds (in {pep}`517` and +{pep}`518`), pip had only supported installing packages using `setup.py` files +that were built using {pypi}`setuptools`. + +The interface documented here is retained currently solely for legacy purposes, +until the migration to `pyproject.toml`-based builds can be completed. + +```{caution} +The arguments and syntax of the various invocations of `setup.py` made by +pip, are considered an implementation detail that is strongly coupled with +{pypi}`setuptools`. This build system interface is not meant to be used by any +other build backend, which should be based on the {doc}`pyproject-toml` build +system interface instead. + +Further, projects should _not_ expect to rely on there being any form of +backward compatibility guarantees around the `setup.py` interface. +``` + +## Build process + +The overall process for building a package is: + +- Generate the package's metadata. +- Generate a wheel for the package. + - If this fails and we're trying to install the package, attempt a direct + installation. + +The wheel can then be used to perform an installation, if necessary. + +### Metadata Generation + +As a first step, `pip` needs to get metadata about a package (name, version, +dependencies, and more). It collects this by calling `setup.py egg_info`. + +The `egg_info` command generates the metadata for the package, which pip can +then consume and proceed to gather all the dependencies of the package. Once +the dependency resolution process is complete, pip will proceed to the next +stage of the build process for these packages. + +### Wheel Generation + +When provided with a {term}`pypug:source distribution (or "sdist")` for a +package, pip will attempt to build a {term}`pypug:wheel`. Since wheel +distributions can be [cached](wheel-caching), this can greatly speed up future +installations for the package. + +This is done by calling `setup.py bdist_wheel` which requires the {pypi}`wheel` +package to be installed. + +If this wheel generation is successful (this can include compiling C/C++ code, +depending on the package), the generated wheel is added to pip's wheel cache +and will be used for this installation. The built wheel is cached locally +by pip to avoid repeated identical builds. + +If this wheel generation fails, pip will attempt a direct installation instead. + +### Direct Installation + +When all else fails, pip will invoke `setup.py install` to install a package +using setuptools' mechanisms to perform the installation. This is currently the +last-resort fallback for projects that cannot be built into wheels, and may not +be supported in the future. + +### Editable Installation + +For installing packages in "editable" mode +({ref}`pip install --editable `), pip will invoke +`setup.py develop`, which will use setuptools' mechanisms to perform an +editable/development installation. + +### Cleanup + +After attempting installation, pip may run `setup.py clean` to clean up build +artifacts from that setuptools has generated. + +## Setuptools Injection + +To support projects that directly use `distutils`, pip injects `setuptools` into +`sys.modules` before invoking `setup.py`. This injection should be transparent +to `distutils`-based projects. + +## Customising the build + +The `--global-option` and `--build-option` arguments to the `pip install` +and `pip wheel` inject additional arguments into the `setup.py` command +(`--build-option` is only available in `pip wheel`). + +```{attention} +The use of `--global-option` and `--build-option` is highly setuptools +specific, and is considered more an accident of the current implementation than +a supported interface. It is documented here for completeness. These flags will +not be supported, once this build system interface is dropped. +``` + +These arguments are included in the command as follows: + +``` +python setup.py BUILD COMMAND +``` + +The options are passed unmodified, and presently offer direct access to the +distutils command line. For example: + +```{pip-cli} +$ pip wheel --global-option bdist_ext --global-option -DFOO wheel +``` + +will result in pip invoking: + +``` +setup.py bdist_ext -DFOO bdist_wheel -d TARGET +``` + +This passes a preprocessor symbol to the extension build. + +(build-output)= + +## Build Output + +Any output produced by the build system will be read by pip (for display to the +user if requested). In order to correctly read the build system output, pip +requires that the output is written in a well-defined encoding, specifically +the encoding the user has configured for text output (which can be obtained in +Python using `locale.getpreferredencoding`). If the configured encoding is +ASCII, pip assumes UTF-8 (to account for the behaviour of some Unix systems). + +Build systems should ensure that any tools they invoke (compilers, etc) produce +output in the correct encoding. In practice - and in particular on Windows, +where tools are inconsistent in their use of the "OEM" and "ANSI" codepages - +this may not always be possible. pip will therefore attempt to recover cleanly +if presented with incorrectly encoded build tool output, by translating +unexpected byte sequences to Python-style hexadecimal escape sequences +(`"\x80\xff"`, etc). However, it is still possible for output to be displayed +using an incorrect encoding (mojibake). diff --git a/docs/html/reference/index.md b/docs/html/reference/index.md index 13e57b2a478..8d690188d7a 100644 --- a/docs/html/reference/index.md +++ b/docs/html/reference/index.md @@ -6,5 +6,6 @@ interoperability standards that pip utilises/implements. ```{toctree} :titlesonly: +build-system/index requirements-file-format ``` diff --git a/docs/html/topics/caching.md b/docs/html/topics/caching.md index 0f4dfe9b90b..d584859c334 100644 --- a/docs/html/topics/caching.md +++ b/docs/html/topics/caching.md @@ -1,6 +1,7 @@ # Caching ```{versionadded} 6.0 + ``` pip provides an on-by-default caching, designed to reduce the amount of time @@ -26,6 +27,8 @@ While this cache attempts to minimize network activity, it does not prevent network access altogether. If you want a local install solution that circumvents accessing PyPI, see {ref}`Installing from local packages`. +(wheel-caching)= + ### Locally built wheels pip attempts to use wheels from its local wheel cache whenever possible. @@ -38,6 +41,10 @@ wheel using the package's build system. If the build is successful, this wheel is added to the cache and used in subsequent installs for the same package version. +Wheels built from source distributions provided to pip as a direct path (such +as `pip install .`) are not cached across runs, though they may be reused within +the same `pip` execution. + ```{versionchanged} 20.0 pip now caches wheels when building from an immutable Git reference (i.e. a commit hash). @@ -79,7 +86,7 @@ implementation detail and may change between any two versions of pip. ## Disabling caching -pip's caching behaviour is disabled by passing the ``--no-cache-dir`` option. +pip's caching behaviour is disabled by passing the `--no-cache-dir` option. It is, however, recommended to **NOT** disable pip's caching. Doing so can significantly slow down pip (due to repeated operations and package builds) diff --git a/docs/html/topics/repeatable-installs.md b/docs/html/topics/repeatable-installs.md index eca633d4ade..c6e8f9689e4 100644 --- a/docs/html/topics/repeatable-installs.md +++ b/docs/html/topics/repeatable-installs.md @@ -94,5 +94,5 @@ identical packages. Beware of the `setup_requires` keyword arg in {file}`setup.py`. The (rare) packages that use it will cause those dependencies to be downloaded by setuptools directly, skipping pip's protections. If you need to use such a -package, see {ref}`Controlling setup_requires `. +package, see {ref}`Controlling setup_requires `. ``` diff --git a/news/10497.doc.rst b/news/10497.doc.rst new file mode 100644 index 00000000000..8c77c32ac66 --- /dev/null +++ b/news/10497.doc.rst @@ -0,0 +1 @@ +Create a "Build System Interface" reference section, for documenting how pip interacts with build systems.