From 0436c0b7194bb7a75b0684fd0246e785b7851342 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sat, 5 Jun 2021 11:55:48 +0200 Subject: [PATCH 01/17] PEP 660: typo --- pep-0660.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0660.rst b/pep-0660.rst index d47f5885c39..dd323aa2421 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -142,7 +142,7 @@ If not defined, the default implementation is equivalent to ``return []``. What to put in the wheel ------------------------ -Build backends may use different techniques to achive the goals of an editable +Build backends may use different techniques to achieve the goals of an editable install. This section provides examples and is not normative. * Build backends may choose to place a ``.pth`` file at the root of the ``.whl`` file, From b201e36bd396c3a73b02a4bac82233e718831e5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sat, 5 Jun 2021 11:57:48 +0200 Subject: [PATCH 02/17] PEP 660: clarify what to put in the wheel --- pep-0660.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/pep-0660.rst b/pep-0660.rst index dd323aa2421..1cd1d054a72 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -142,6 +142,7 @@ If not defined, the default implementation is equivalent to ``return []``. What to put in the wheel ------------------------ +Build backends must populate the generated wheel with files that when installed will result in an editable install. Build backends may use different techniques to achieve the goals of an editable install. This section provides examples and is not normative. From 654ba54436cd292aa47bcf2268d2e1218e170b3c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sat, 5 Jun 2021 12:23:46 +0200 Subject: [PATCH 03/17] PEP 660: mention symbolic links --- pep-0660.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/pep-0660.rst b/pep-0660.rst index 1cd1d054a72..8a8a31ca5bf 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -158,6 +158,11 @@ install. This section provides examples and is not normative. a path importable, often including the project's own ``setup.py`` and other scripts that would not be part of a normal installation. The proxy strategy can achieve a higher level of fidelity than path-based methods. +* Symbolic links are another useful mechanism to realize editable installs. + Since, at the time this writing, the ``wheel`` specification does not support + symbolic links, they are not directly usable. It is however possible to defer + the creation of symbolic links to an import hook or executable ``.pth`` that + the backend adds in the “editable” wheel. Frontend requirements --------------------- From 76d9186df935afa727ce791f8d550c4d3b32e49e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sat, 5 Jun 2021 12:37:27 +0200 Subject: [PATCH 04/17] PEP 660: add "virtual wheels" to rejected ideas --- pep-0660.rst | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/pep-0660.rst b/pep-0660.rst index 8a8a31ca5bf..82975aab230 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -195,12 +195,40 @@ cached nor distributed. Rejected ideas ============== +``editable`` local version identifier +------------------------------------- + The ideas of having build backends append or modify the local version identifier to include the ``editable`` string has been rejected because it would not satisfy ``==`` version speicifier that include the local version identifier. In other workds ``pkg==1.0+local`` is not satisfied by version ``1.0+local.editable``. +Virtual wheel +------------- + +`Another approach was proposed `_, where +the build backend returns a mapping from source files and directories to the +installed layout. It is then up to the installer frontend to realize the +editable installation by whatever mean it deems adequate for its users. + +At the time of writing this PEP, it is clear that the community has a wide +range of theoretical and practical expectations about editable installs. The +reality is that the only one there is wide experience with is path insertion +via .pth (i.e. what setup.py develop does). + +We believe that PEP 660 better addresses these "unknown unknowns" today in by +letting project authors select or implement the backend that provides the +editable mechanism that best suit their requirements. + +Since the frontend has no latitude in *how* to install the "editable" wheel, in +case of issue, there is only one place to investigate: the build backend. With +the virtual wheel proposal, issues need to be investigated in the frontend, the +backend and possiblty the specification. There is also a high probability that +frontends, implementing the specification in different ways, will produce +installation that behave differently than project authors intended, creating +confusion, or worse, projects that only work with specific frontends. + References ========== From d606a9d3c3d3342b9e501c1415bbbff86fbf244a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sat, 5 Jun 2021 12:53:47 +0200 Subject: [PATCH 05/17] PEP 660: consistency in referring to "editable" wheels --- pep-0660.rst | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index 82975aab230..5e640bd9bf5 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -114,11 +114,11 @@ Build-backends must produce wheels that have the same dependencies with the exception that they can add dependencies necessary for their editable mechanism to function at runtime (such as `editables`_). -The filename for the “editable” wheel needs to be PEP 427 compliant too. It +The filename for the "editable" wheel needs to be PEP 427 compliant too. It does not need to use the same tags as ``build_wheel`` but it must be tagged as compatible with the system. -An “editable” wheel uses the wheel format not for distribution but as ephemeral +An "editable" wheel uses the wheel format not for distribution but as ephemeral communication between the build system and the front end. This avoids having the build backend install anything directly. This wheel must not be exposed to end users, nor cached, nor distributed. @@ -162,12 +162,12 @@ install. This section provides examples and is not normative. Since, at the time this writing, the ``wheel`` specification does not support symbolic links, they are not directly usable. It is however possible to defer the creation of symbolic links to an import hook or executable ``.pth`` that - the backend adds in the “editable” wheel. + the backend adds in the "editable" wheel. Frontend requirements --------------------- -Frontends must install editable wheels in the same way as regular wheels. +Frontends must install "editable" wheels in the same way as regular wheels. This also means uninstallation of editables does not require any special treatment. Frontends must create a ``direct_url.json`` file in the ``.dist-info`` @@ -212,22 +212,26 @@ the build backend returns a mapping from source files and directories to the installed layout. It is then up to the installer frontend to realize the editable installation by whatever mean it deems adequate for its users. +In terms of capabilities, both proposals seem equivalent, at least in +principle. + At the time of writing this PEP, it is clear that the community has a wide range of theoretical and practical expectations about editable installs. The reality is that the only one there is wide experience with is path insertion via .pth (i.e. what setup.py develop does). -We believe that PEP 660 better addresses these "unknown unknowns" today in by -letting project authors select or implement the backend that provides the -editable mechanism that best suit their requirements. - +We believe that PEP 660 better addresses these "unknown unknowns" today in the +most reliable way, in by letting project authors select or implement the +backend that provides the editable mechanism that best suit their requirements. Since the frontend has no latitude in *how* to install the "editable" wheel, in -case of issue, there is only one place to investigate: the build backend. With -the virtual wheel proposal, issues need to be investigated in the frontend, the -backend and possiblty the specification. There is also a high probability that -frontends, implementing the specification in different ways, will produce -installation that behave differently than project authors intended, creating -confusion, or worse, projects that only work with specific frontends. +case of issue, there is only one place to investigate: the build backend. + +With the competing proposal, issues need to be investigated in the frontend, +the backend and possiblty the specification. There is also a high probability +that different frontends, implementing the specification in different ways, +will produce installations that behave differently than project authors +intended, creating confusion, or worse, projects that only work with specific +frontends. References ========== From b28203ce075653699826c698cdc9925e6791150a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sat, 5 Jun 2021 18:52:48 +0200 Subject: [PATCH 06/17] PEP 660: typos --- pep-0660.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index 5e640bd9bf5..72788daa88b 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -201,7 +201,7 @@ Rejected ideas The ideas of having build backends append or modify the local version identifier to include the ``editable`` string has been rejected because it would not satisfy ``==`` version speicifier that include the local version -identifier. In other workds ``pkg==1.0+local`` is not satisfied by version +identifier. In other words ``pkg==1.0+local`` is not satisfied by version ``1.0+local.editable``. Virtual wheel @@ -221,7 +221,7 @@ reality is that the only one there is wide experience with is path insertion via .pth (i.e. what setup.py develop does). We believe that PEP 660 better addresses these "unknown unknowns" today in the -most reliable way, in by letting project authors select or implement the +most reliable way, by letting project authors select or implement the backend that provides the editable mechanism that best suit their requirements. Since the frontend has no latitude in *how* to install the "editable" wheel, in case of issue, there is only one place to investigate: the build backend. From 399db6729b7f187df63e60dd4920e73f3607a903 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Mon, 14 Jun 2021 14:04:38 +0200 Subject: [PATCH 07/17] PEP 660: add another suggestion about symbolic links --- pep-0660.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index 72788daa88b..11b1c846dc2 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -160,9 +160,12 @@ install. This section provides examples and is not normative. can achieve a higher level of fidelity than path-based methods. * Symbolic links are another useful mechanism to realize editable installs. Since, at the time this writing, the ``wheel`` specification does not support - symbolic links, they are not directly usable. It is however possible to defer - the creation of symbolic links to an import hook or executable ``.pth`` that - the backend adds in the "editable" wheel. + symbolic links, they are not directly usable. It is however possible for the + backend to create an symlink structure in a build directory of the source + tree such as ``.editable/$COMPATIBILITY_TAGS``, and add that directory to the + python path via a ``.pth`` file in the "editable" wheel. Alternatively it is + also possible to defer the creation of symbolic links to an import hook or + executable ``.pth`` that the backend adds in the "editable" wheel. Frontend requirements --------------------- From 53e4ebef1d2dbca844f70e6f56fb6fc73998816b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Mon, 14 Jun 2021 14:06:52 +0200 Subject: [PATCH 08/17] PEP 660: add a clarification about symbolic links --- pep-0660.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pep-0660.rst b/pep-0660.rst index 11b1c846dc2..c54104c2553 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -165,7 +165,8 @@ install. This section provides examples and is not normative. tree such as ``.editable/$COMPATIBILITY_TAGS``, and add that directory to the python path via a ``.pth`` file in the "editable" wheel. Alternatively it is also possible to defer the creation of symbolic links to an import hook or - executable ``.pth`` that the backend adds in the "editable" wheel. + executable ``.pth`` that the backend adds in the "editable" wheel (although + this approach requires special care to keep the RECORDS file up-to-date). Frontend requirements --------------------- From a3b34e5cab0ba74aa145de26e108f7a0fb9b1943 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 11:57:23 +0200 Subject: [PATCH 09/17] PEP 660: fix typo --- pep-0660.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0660.rst b/pep-0660.rst index c54104c2553..f0e50f3c346 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -214,7 +214,7 @@ Virtual wheel `Another approach was proposed `_, where the build backend returns a mapping from source files and directories to the installed layout. It is then up to the installer frontend to realize the -editable installation by whatever mean it deems adequate for its users. +editable installation by whatever means it deems adequate for its users. In terms of capabilities, both proposals seem equivalent, at least in principle. From b3d741dd1777132b853be3ad9ab1e782f47db5d3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 11:57:31 +0200 Subject: [PATCH 10/17] PEP 660: clarify symlink support --- pep-0660.rst | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index f0e50f3c346..b33aa755895 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -160,13 +160,14 @@ install. This section provides examples and is not normative. can achieve a higher level of fidelity than path-based methods. * Symbolic links are another useful mechanism to realize editable installs. Since, at the time this writing, the ``wheel`` specification does not support - symbolic links, they are not directly usable. It is however possible for the - backend to create an symlink structure in a build directory of the source - tree such as ``.editable/$COMPATIBILITY_TAGS``, and add that directory to the - python path via a ``.pth`` file in the "editable" wheel. Alternatively it is - also possible to defer the creation of symbolic links to an import hook or - executable ``.pth`` that the backend adds in the "editable" wheel (although - this approach requires special care to keep the RECORDS file up-to-date). + symbolic links, they are not directly usable to set-up symbolic links in the + target environment. It is however possible for the backend to create a + symlink structure in some ``build`` directory of the source tree, and add + that directory to the python path via a ``.pth`` file in the "editable" + wheel. If some files linked in this manner depend on python implementation or + version, ABI or platform, care must be taken to generate the link structure + in different directories depending on compatibility tags, so the same project + tree can be installed in editable mode in multiple environments. Frontend requirements --------------------- From 01b8c1550637f0a59f24e016ccb20c142a97ea41 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 12:17:37 +0200 Subject: [PATCH 11/17] PEP 660: add unpacked wheel to rejected ideas --- pep-0660.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/pep-0660.rst b/pep-0660.rst index b33aa755895..4b4de3b92ca 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -238,6 +238,16 @@ will produce installations that behave differently than project authors intended, creating confusion, or worse, projects that only work with specific frontends. +Unpacked wheel +-------------- + +A `prototype `_ was made that +created an unpacked wheel in a temporary directory, to be copied to the target +environment by the frontend. This approach was not pursued because a wheel +archive is easy to create for the backend, and using a wheel as communication +mechanism is a better fit with the PEP 517 philosophy, and therefore keeps +things simpler for the frontend. + References ========== From 563065222f94cfcdcd264a5c1770b3ad7be2b4c1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 12:21:16 +0200 Subject: [PATCH 12/17] PEP 660: clarify expectations of an editable install --- pep-0660.rst | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index 4b4de3b92ca..aa42f4a9883 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -67,11 +67,14 @@ require a compilation and/or installation step to become effective. The exact steps to perform will remain specific to the build backend used. When a project is installed in editable mode, users expect the installation to -behave identically as a regular installation. Depending on the way build -backends implement this specification, some minor differences may be visible -such as the presence of additional files that are in the source tree and would -not be part of a regular install. Build backends are encouraged to document -such potential differences. +behave identically as a regular installation. In particular the code must be +importable by other code, and metadata must be available to standard mechanisms +such as ``importlib.metadata``. + +Depending on the way build backends implement this specification, some minor +differences may be visible such as the presence of additional files that are in +the source tree and would not be part of a regular install. Build backends are +encouraged to document such potential differences. The Mechanism ============= From ad2d7bbf5cc7d7972a95415a0b33997bca83a214 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 12:37:09 +0200 Subject: [PATCH 13/17] PEP 660: mention the .data directory --- pep-0660.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/pep-0660.rst b/pep-0660.rst index aa42f4a9883..fef4eebbf86 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -200,6 +200,16 @@ Frontends must not expose the wheel obtained from ``build_wheel_for_editable`` to end users. The wheel must be discarded after installation and must not be cached nor distributed. +Limitations +=========== + +This PEP does not support making the content of the wheel .data directory +"editable" (scripts, C header files, and other data files). + +Scripts in the .data directory are therefore not supported by this PEP either, +and package authors are encouraged to migrate to ``console_scripts`` entry +points, or make their scripts tiny wrapper around library functionality. + Rejected ideas ============== From f5c5a654b85d8d5c551d30458a9fea78e342f175 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 12:53:43 +0200 Subject: [PATCH 14/17] PEP 660: clarify differences with PEP 662 --- pep-0660.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index fef4eebbf86..f76b09a7fa2 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -225,13 +225,18 @@ identifier. In other words ``pkg==1.0+local`` is not satisfied by version Virtual wheel ------------- -`Another approach was proposed `_, where +Another approach was proposed in PEP 662, where the build backend returns a mapping from source files and directories to the installed layout. It is then up to the installer frontend to realize the editable installation by whatever means it deems adequate for its users. -In terms of capabilities, both proposals seem equivalent, at least in -principle. +In terms of capabilities, both proposals provide the core "editable" feature. + +The key difference is that PEP 662 leaves it to the frontend to decide how the +editable installation will be realized, while with this PEP, the choice must be +made by the backend. Both approaches can in principle provide several editable +installation methods for a given project, and let the developer choose one at +install time. At the time of writing this PEP, it is clear that the community has a wide range of theoretical and practical expectations about editable installs. The @@ -239,17 +244,18 @@ reality is that the only one there is wide experience with is path insertion via .pth (i.e. what setup.py develop does). We believe that PEP 660 better addresses these "unknown unknowns" today in the -most reliable way, by letting project authors select or implement the -backend that provides the editable mechanism that best suit their requirements. -Since the frontend has no latitude in *how* to install the "editable" wheel, in -case of issue, there is only one place to investigate: the build backend. +most reliable way, by letting project authors select the backend or implement +the method that provides the editable mechanism that best suit their +requirements, and test it works correctly. Since the frontend has no latitude +in *how* to install the "editable" wheel, in case of issue, there is only one +place to investigate: the build backend. -With the competing proposal, issues need to be investigated in the frontend, +With PEP 662, issues need to be investigated in the frontend, the backend and possiblty the specification. There is also a high probability that different frontends, implementing the specification in different ways, will produce installations that behave differently than project authors intended, creating confusion, or worse, projects that only work with specific -frontends. +frontends or IDEs. Unpacked wheel -------------- From 137fe99cbb3fc40cffd2e0410587c1bbbc91ffad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 12:54:37 +0200 Subject: [PATCH 15/17] PEP 660: remove the schem argument --- pep-0660.rst | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index f76b09a7fa2..6b7283e1c0b 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -88,17 +88,9 @@ build_wheel_for_editable :: - def build_wheel_for_editable( - wheel_directory, - scheme=scheme, - config_settings=None): + def build_wheel_for_editable(wheel_directory, config_settings=None): ... -``scheme``: a dictionary of installation categories ``{ 'purelib': -'/home/py/.../site-packages', 'platlib': '...'}``. This makes it possible to -use relative paths to the source code, which might help the interpreter find -the package after the root path changes with ``chroot`` or similar. - Must build a ``.whl`` file, and place it in the specified ``wheel_directory``. It must return the basename (not the full path) of the .whl file it creates, as a unicode string. From fc87753a85522bc4b5005b3bf1131925234cee1c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 20 Jun 2021 20:10:43 +0200 Subject: [PATCH 16/17] PEP 660: further clarify the limitation wrt the .data dir --- pep-0660.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/pep-0660.rst b/pep-0660.rst index 6b7283e1c0b..4f495c5941d 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -195,12 +195,12 @@ cached nor distributed. Limitations =========== -This PEP does not support making the content of the wheel .data directory -"editable" (scripts, C header files, and other data files). - -Scripts in the .data directory are therefore not supported by this PEP either, -and package authors are encouraged to migrate to ``console_scripts`` entry -points, or make their scripts tiny wrapper around library functionality. +With regard to the wheel ``.data`` directory, this PEP focuses on making the +``purelib`` and ``platlib`` categories (installed into site-packages) +"editable". It does not make special provision for the other categories such as +``headers``, ``data`` and ``scripts``. Package authors are encouraged to use +``console_scripts``, make their ``scripts`` tiny wrappers around library +functionality, or manage these from the source checkout during development. Rejected ideas ============== From 689e4ebeb34d39486c3bfc17496ea5cbcf967f80 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Mon, 21 Jun 2021 11:39:10 +0200 Subject: [PATCH 17/17] PEP 660: update title --- pep-0660.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0660.rst b/pep-0660.rst index 4f495c5941d..a0eed42c1f8 100644 --- a/pep-0660.rst +++ b/pep-0660.rst @@ -1,5 +1,5 @@ PEP: 660 -Title: Editable installs for PEP 517 style build backends +Title: Editable installs for pyproject.toml based builds Author: Daniel Holth , Stéphane Bidoul Sponsor: Paul Moore Discussions-To: https://discuss.python.org/t/draft-pep-editable-installs-for-pep-517-style-build-backends/8510