During poetry build an installed binary from build-system.requires is not available

[RobertoRoos]

Description

When running poetry build to build a package distributable, installed executable are not available on PATH.

This behavior is different during poetry install: here binaries from build requires are available during build.

I have the following build script:

# build_package.py
import subprocess


def main():
    print("Running `pyside6-uic --help`...")
    result = subprocess.run(
        ["pyside6-uic", "--help"],
        check=True,  # Raise exception on failure
        capture_output=True,
    )


if __name__ == "__main__":
    main()

The pyproject.toml is below.

I get an error (see below) during build, but not during install.

Workarounds

The build behavior is correct using python -m build.
So a workaround could be running:

poetry run python -m build

Another workaround is to install the package providing the binary to the project venv, and running Poetry from there:

$ .venv\Scripts\Activate.ps1
(.venv) $ pip install pyside6-essentials
(.venv) $ poetry build ...

Poetry Installation Method

install.python-poetry.org

Operating System

Windows 10

Poetry Version

Poetry (version 1.8.3)

Poetry Configuration

cache-dir = "C:\\Users\\ROQ\\AppData\\Local\\pypoetry\\Cache"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
packages/pypi"
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs"  # C:\Users\ROQ\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

Platform: "win-amd64"
Python version: "3.11"
Current installation scheme: "nt"

Paths: 
        data = "C:\Program Files\Python311"
        include = "C:\Program Files\Python311\Include"
        platinclude = "C:\Program Files\Python311\Include"
        platlib = "C:\Program Files\Python311\Lib\site-packages"
        platstdlib = "C:\Program Files\Python311\Lib"
        purelib = "C:\Program Files\Python311\Lib\site-packages"
        scripts = "C:\Program Files\Python311\Scripts"
        stdlib = "C:\Program Files\Python311\Lib"

Variables: 
        BINDIR = "C:\Program Files\Python311"
        BINLIBDEST = "C:\Program Files\Python311\Lib"
        EXE = ".exe"
        EXT_SUFFIX = ".cp311-win_amd64.pyd"
        INCLUDEPY = "C:\Program Files\Python311\Include"
        LIBDEST = "C:\Program Files\Python311\Lib"
        TZPATH = ""
        VERSION = "311"
        VPATH = "..\.."
        abiflags = ""
        base = "C:\Program Files\Python311"
        exec_prefix = "C:\Program Files\Python311"
        installed_base = "C:\Program Files\Python311"
        installed_platbase = "C:\Program Files\Python311"
        platbase = "C:\Program Files\Python311"
        platlibdir = "DLLs"
        prefix = "C:\Program Files\Python311"
        projectbase = "C:\Program Files\Python311"
        py_version = "3.11.8"
        py_version_nodot = "311"
        py_version_nodot_plat = "311"
        py_version_short = "3.11"
        srcdir = "C:\Program Files\Python311"
        userbase = "C:\Users\ROQ\AppData\Roaming\Python"

Example pyproject.toml

[tool.poetry]
name = "my-package"
version = "0.1.0"
description = ""
authors = ["Robert Roos <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"

[build-system]
requires = [
    "poetry-core",
    "PySide6-Essentials ~= 6.7.2",
]
build-backend = "poetry.core.masonry.api"

[tool.poetry.build]
script = "build_package.py"

Poetry Runtime Logs

FileNotFoundError: [WinError 2] The system cannot find the file specified

  CalledProcessError

  Command '['C:/Temp/tmp14n85r8e/.venv/Scripts/python.exe', 'build_package.py']' returned non-zero exit status 1.

[dimbleby]

duplicate #8807 presumably

[RobertoRoos]

To debug this, I've added a print and a sleep in the build script:

    subprocess.run(["cmd.exe", "/c", "echo", "%PATH%"])
    print(sys.path)
    sleep(30.0)

---

Running poetry install -v now shows:

C:\Temp\tmptc2h5ep9.venv\Scripts;C:\Program Files\PowerShell\7; ...
['C:\Users\ROQ\PycharmProjects\pythonProject1', 'C:\Program Files\Python311\python311.zip', 'C:\Program Files\Python311\DLLs', 'C:\Program Files\Python311\Lib', 'C:\Program Files\Python311', 'C:\Temp\tmptc2h5ep9\.venv', 'C:\Temp\tmptc2h5ep9\.venv\Lib\site-packages']

And (while the install command is hanging) I see the file C:\Temp\tmptc2h5ep9\.venv\Scripts\pyside6-uic.exe does exist.

---

Running poetry build -f wheel -v shows:

C:\Program Files\PowerShell\7; ...
['C:\Users\ROQ\PycharmProjects\pythonProject1', 'C:\Program Files\Python311\python311.zip', 'C:\Program Files\Python311\DLLs', 'C:\Program Files\Python311\Lib', 'C:\Program Files\Python311', 'C:\Temp\tmpuvxz5_s0\.venv', 'C:\Temp\tmpuvxz5_s0\.venv\Lib\site-packages']

There is no Scripts/ directory, but the file C:\temp\tmpuvxz5_s0\.venv\Scripts\pyside6-uic.exe is there.

---

sys.path doesn't show the Scripts directory at all, hence I'm printing PATH from the OS also.

[RobertoRoos]

duplicate #8807 presumably

Yeah, looks like it.

I'm updating my comment above once more, then we can close this as duplicate if that's better.

EDIT: I just wanted to emphasize the requirements are honored, in this example PySide6_Essentials is available in the temp venv, and adding

import PySide6

...to package_build.py success in both install and build.

[RobertoRoos]

Interesting. Installing pyside6-essentials to the venv for this project and running poetry build ... with the venv activated does make the build succeed.

System PATH, printed from build_package.py, looks like:

C:\temp\tmpd7pgbi6j.venv\Lib\site-packages\PySide6;C:\Users\ROQ\PycharmProjects\pythonProject1.venv/Scripts;C:\Program Files\PowerShell\7;...

So the binary from the project venv ends up being used.

But activating the venv is essential here. Doing poetry add ... ; poetry build ... without activating the venv still gives the same errors as above.

[Secrus]

@RobertoRoos would you be able to put your discoveries into one message and post it in the #8807 issue, so that we can close this?

[RobertoRoos]

See, see summary in #8807 (comment)

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.