Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build backend: Add --list option #9610

Merged
merged 4 commits into from
Dec 4, 2024
Merged

Build backend: Add --list option #9610

merged 4 commits into from
Dec 4, 2024

Conversation

konstin
Copy link
Member

@konstin konstin commented Dec 3, 2024 •
edited
Loading

Add the uv build --list, a "subcommand" to list the files that would be included when building a distribution. It does not build the distribution, except when a source dist is required for source dist -> wheel. This is an important debugging tool for the include and exclude options: Did i actually include the files I wanted, or am i shipping a broken distribution? Are there any temporary files I still need to exclude?

Cargo offers this as cargo package --list.

--list is preview-exclusive, since it requires the fast path, which I also put into preview.

Examples:

image

image

image

I'll fix the error handling in a follow-up.

Tagging as enhancement because it changes the stable output slightly (two lines instead of one).

CC @charliermarsh for uv-wide consistency in the stdout/stderr handling.

@konstin konstin added the preview Experimental behavior label Dec 3, 2024
@konstin konstin force-pushed the konsti/build-backend-list-cli branch from bfc7af9 to 58d621b Compare December 3, 2024 16:33
@konstin konstin added enhancement New feature or improvement to existing functionality preview Experimental behavior and removed preview Experimental behavior labels Dec 3, 2024
BurntSushi
BurntSushi approved these changes Dec 3, 2024
View reviewed changes
Copy link
Member

@BurntSushi BurntSushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes sense to me. Although I am not sure I have enough knowledge to understand why --list requires the fast path. Will there be other options that also require the fast path?

(It occurs to me that "fast path" may be a misnomer, or at least, selling itself short. Because it isn't just a fast path but a required mode in order to access additional functionality.)

crates/uv-cli/src/lib.rs
/// build the wheel.
///
/// This option can be combined with `--sdist` and `--wheel` for inspecting different build
/// paths.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you mention in the docs here that this conflicts with --force-pep517? I might even spare a sentence or two saying why that conflict exists. (I'm guessing that in the fast path, it's easier to have a side channel asking the build backend to emit extra output? I haven't read the rest of the PR yet.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added an explanation about how this doesn't go through PEP 517.

@konstin
Copy link
Member Author

konstin commented Dec 3, 2024

Although I am not sure I have enough knowledge to understand why --list requires the fast path. Will there be other options that also require the fast path?

Potentially - We may grow more uv build backend only features in the future that don't work through the limited set of PEP 517 hooks. "direct build" is a better term now than fast path. We need to explain to users that uv build can be both the PEP 517 generic source dist and wheel builder (python -m build, pip wheel, etc.) and the CLI command to the uv build system (poetry build, hatch build, maturin build). The latter has more options than the former (maturin build is a good example), while the former is the "official" way, there is some agreement that python -m build is the "official" way to build. By default, uv will be the latter for the uv build backend, but we allow switching to the former with --force-pep517, since this is what your package's users will experience.

@charliermarsh
Copy link
Member

I think it's correct for the output to go to stderr in general, and then stdout with --list.

konstin added a commit that referenced this pull request Dec 3, 2024
@konstin
Rename: Fast path -> direct build
302543c 
See discussion at #9610 (comment)
konstin added a commit that referenced this pull request Dec 3, 2024
@konstin
Build backend: Add direct builds to the resolver and installer
b927433 
This is like #9556, but at the level of all other builds, including the resolver and installer. Going through PEP 517 to build a package is slow, so when building a package with the uv build backend, we can call into the uv build backend directly instead: No temporary virtual env, no temp venv sync, no python subprocess calls, no uv subprocess calls.

This fast path is gated through preview. Since the uv wheel is not available at test time, I've manually confirmed the feature by comparing `uv venv && cargo run pip install . -v --preview --reinstall .` and `uv venv && cargo run pip install . -v --reinstall .`.

Do we need a global option to disable the fast path? There is one for `uv build` because `--force-pep517` moves `uv build` much closer to a `pip install` from source that a user of a library would experience.See discussion at #9610 (comment)
This was referenced Dec 3, 2024
Merged
Open
@konstin konstin merged commit 7d82cbf into main Dec 4, 2024
63 of 64 checks passed
@konstin konstin deleted the konsti/build-backend-list-cli branch December 4, 2024 08:52
konstin added a commit that referenced this pull request Dec 4, 2024
@konstin
Rename: Fast path -> direct build
7fa893d 
See discussion at #9610 (comment)
konstin added a commit that referenced this pull request Dec 4, 2024
@konstin
Build backend: Add direct builds to the resolver and installer
87823cf 
This is like #9556, but at the level of all other builds, including the resolver and installer. Going through PEP 517 to build a package is slow, so when building a package with the uv build backend, we can call into the uv build backend directly instead: No temporary virtual env, no temp venv sync, no python subprocess calls, no uv subprocess calls.

This fast path is gated through preview. Since the uv wheel is not available at test time, I've manually confirmed the feature by comparing `uv venv && cargo run pip install . -v --preview --reinstall .` and `uv venv && cargo run pip install . -v --reinstall .`.

Do we need a global option to disable the fast path? There is one for `uv build` because `--force-pep517` moves `uv build` much closer to a `pip install` from source that a user of a library would experience.See discussion at #9610 (comment)
@BurntSushi
Copy link
Member

Yeah I like "direct build" a lot better. :-) Otherwise makes sense, thank you for explaining!

konstin added a commit that referenced this pull request Dec 4, 2024
@konstin
Build backend: Add direct builds to the resolver and installer (#9621)
6ed6fc1 
This is like #9556, but at the level of all other builds, including the
resolver and installer. Going through PEP 517 to build a package is
slow, so when building a package with the uv build backend, we can call
into the uv build backend directly instead: No temporary virtual env, no
temp venv sync, no python subprocess calls, no uv subprocess calls.

This fast path is gated through preview. Since the uv wheel is not
available at test time, I've manually confirmed the feature by comparing
`uv venv && cargo run pip install . -v --preview --reinstall .` and `uv
venv && cargo run pip install . -v --reinstall .`. When hacking the
preview so that the python uv build backend works without the setting
the direct build also (wheel built with `maturin build --profile
profiling`), we can see the perfomance difference:

```
$ hyperfine --prepare "uv venv" --warmup 3 \
    "UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview" \
    "target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/"
Benchmark 1: UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview
  Time (mean ± σ):      33.1 ms ±   2.5 ms    [User: 25.7 ms, System: 13.0 ms]
  Range (min … max):    29.8 ms …  47.3 ms    73 runs
 
Benchmark 2: target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/
  Time (mean ± σ):     115.1 ms ±   4.3 ms    [User: 54.0 ms, System: 27.0 ms]
  Range (min … max):   109.2 ms … 123.8 ms    25 runs
 
Summary
  UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview ran
    3.48 ± 0.29 times faster than target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/
```

Do we need a global option to disable the fast path? There is one for
`uv build` because `--force-pep517` moves `uv build` much closer to a
`pip install` from source that a user of a library would experience (See
discussion at #9610), but uv overall doesn't really make guarantees
around the build env of dependencies, so I consider the direct build a
valid option.

Best reviewed commit-by-commit, only the last commit is the actual
implementation, while the preview mode introduction is just a
refactoring touching too many files.
@BrewTestBot BrewTestBot mentioned this pull request Dec 7, 2024
Merged
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Dec 9, 2024
@tmeijn
build(deps): update astral-sh/uv to v0.5.7
840e749 
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.5.6` -> `0.5.7` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.5.7`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#057)

[Compare Source](astral-sh/uv@0.5.6...0.5.7)

##### Enhancements

-   Ignore dynamic version in source dist ([#&#8203;9549](astral-sh/uv#9549))
-   Improve build frontend error handling ([#&#8203;9611](astral-sh/uv#9611))
-   Un-hide `uv build --no-build-logs` option ([#&#8203;9642](astral-sh/uv#9642))
-   Flag version mismatch between sdist and wheel during `uv build` ([#&#8203;9633](astral-sh/uv#9633))
-   Improve message when updater receipt is for a different uv executable ([#&#8203;9487](astral-sh/uv#9487))
-   Add environment variable to disable writing installer metadata files ([#&#8203;8877](astral-sh/uv#8877))
-   Add managed downloads for the latest CPython releases: `3.9.21`, `3.10.16`, `3.11.11`, `3.12.8`, and `3.13.1` ([#&#8203;9696](astral-sh/uv#9696))

##### Preview features

-   Build backend: Add hint on import with preview disabled ([#&#8203;9691](astral-sh/uv#9691))
-   Build backend: Add direct builds to the resolver and installer ([#&#8203;9621](astral-sh/uv#9621))
-   Build backend: Add integration test for scripts ([#&#8203;9635](astral-sh/uv#9635))
-   Build backend: Add template to `uv init` ([#&#8203;9661](astral-sh/uv#9661))
-   Build backend: Add `--list` option ([#&#8203;9610](astral-sh/uv#9610))

##### Bug fixes

-   Create missing parent directories for output file of `uv export` / `uv pip compile` ([#&#8203;9648](astral-sh/uv#9648))
-   Fix missing display of non-freethreaded Python 3.13 in `python list` ([#&#8203;9669](astral-sh/uv#9669))
-   Implement `Ord` and `PartialOrd` without origin for `Requirement` ([#&#8203;9624](astral-sh/uv#9624))
-   Include more sources to avoid lowest bound warning ([#&#8203;9644](astral-sh/uv#9644))
-   Respect build tag priority in `uv.lock` ([#&#8203;9677](astral-sh/uv#9677))

##### Documentation

-   Add `build-essentials` note to build failures doc ([#&#8203;9641](astral-sh/uv#9641))
-   Add entry-point for distroless image in GitLab documentation ([#&#8203;9093](astral-sh/uv#9093))
-   Add documentation for `uv python pin` without a `REQUEST` argument ([#&#8203;9631](astral-sh/uv#9631))
-   Add a link to `uv python pin` reference docs ([#&#8203;9630](astral-sh/uv#9630))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BurntSushi BurntSushi BurntSushi approved these changes

@charliermarsh charliermarsh Awaiting requested review from charliermarsh

Assignees
No one assigned
Labels
enhancement New feature or improvement to existing functionality preview Experimental behavior
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@konstin @charliermarsh @BurntSushi