Skip existing, second iteration: Check the index before uploading #8531

konstin · 2024-10-24T15:59:23Z

The user can pass --check-url <index url>. For each file given to upload, uv checks whether the file is already on the index. If the filename does not exist, uv will upload the file. If the filename does exist and the file on the index is the exact same as the local file (hash match), we skip the upload as there's nothing to do. If the file exists and mismatches with different content, we error: There is an inconsistency, and uv does not upload in this case.

The upload itself may then error or succeed. If it errors, we check the index URL again: Maybe there was a parallel upload of the same file, and the other upload was quicker (avoid TOCTOU errors). We then apply the same logic: If the filename does exist and the file on the index is the same (hash match), it's ok, there was an upload race. If the file exists and mismatches, we error: There is an inconsistency, and uv does not upload in this case. I'm not sure yet if we should do the second check only if the error is one of the twine-recognized file-already-existed conditions; i tend to think we can just do the second check on any error.

This enables re-trying publish, but it still requires the author of a package to ensure that they only try to publish each version from a specific commit. We impose this requirement to avoid situations where different files in a version were built from different sources, which breaks a fundamental assumption in uv.

The name --check-url is up for bike shedding. I'm hesitant to call it --index because it's different from --index in resolve or install commands (only one index, caching turned off).

The test script previously only did step 1, now it also checks skip existing behavior.

An upload with a fresh version succeeds.
If we're using PyPI, uploading the same files again succeeds.
Skip existing works and reports the files as skipped.

Follow-up: Test that skip existing error when the hashes mismatch.

Fixes #7917

crates/uv-client/src/registry_client.rs

konstin · 2024-10-24T16:01:40Z

crates/uv-client/src/registry_client.rs

@@ -143,6 +143,27 @@ impl<'a> RegistryClientBuilder<'a> {
            timeout,
        }
    }
+
+    /// Share the underlying client between two different middleware configurations.
+    pub fn wrap_existing(self, existing: &BaseClient) -> RegistryClient {


Avoid the cost for creating another client.

konstin · 2024-10-24T16:02:52Z

crates/uv-publish/src/lib.rs

+
+    debug!("Checking for {filename} in the registry");
+    let response = registry_client
+        .simple(filename.name(), Some(index_url), index_capabilities)


Do we need the index url here or will it do the right thing without it?

It depends. Was the client created with only this index as the set of locations? It seems right to pass it in though.

konstin · 2024-10-24T16:03:50Z

crates/uv-publish/src/lib.rs

@@ -620,39 +730,11 @@ async fn handle_response(registry: &Url, response: Response) -> Result<bool, Pub
        ));
    }

-    // Detect existing file errors the way twine does.


We scrap the index-specific twine code entirely and do the skip existing checks for any bad status

charliermarsh · 2024-10-28T12:33:25Z

crates/uv-cli/src/lib.rs

+    ///
+    /// The index must provide one of the supported hashes (SHA-256, SHA-384, or SHA-512).
+    #[arg(long,env = EnvVars::UV_PUBLISH_SKIP_EXISTING)]
+    pub skip_existing: Option<IndexUrl>,


Why does this take an index, rather than a bool?

It's the story documented in #7917 (comment) and #7917 (comment): An index may react arbitrarily to a file with a filename that does already exist: PyPI for example checks if the hash matches and then return a plain OK as for a fresh upload, while other registries error even when re-uploading the same file. By proactively checking the simple index over relying on unspecified, inconsistent behaviour, uv publish behaves consistently across registries, while also preventing one case of uploading inconsistent wheel/source dist combinations. In cases where we skip an upload, it is also faster than a twine-style skip existing bool.

But why do we need to provide the index here? Because you need to be able to query for metadata, and the publish URL isn't sufficient?

There could e.g. be multiple index URLs, and we can't compute the index URL from the publish URL, so you need to manually provide the index URL that matches the publish URL. Making it explicit here seemed like a clear interface.

I'm not overly attached to doing it this way, i mainly picked it because it seemed like a clear, explainable design.

Won't this be surprising when coming from another tool like twine where this is a bool? Should we call it something else?

As I've said above, the name is open for bike shedding. I like it for being self explaining (uv publish --skip-existing https://example.org/simple means to skip any files existing on https://example.org/simple when publishing) and for not being confused with the regular index API.

crates/uv-cli/src/lib.rs

zanieb · 2024-10-28T21:49:32Z

It'd be nice if we could re-use the new index settings to improve the publishing experience for uv-managed projects (which I think is the key use-case we should be focusing on). For example:

[[index]]
name = "example"
url = "https://pypi.org/simple"
publish-url = "https://upload.pypi.org/legacy"

with uv publish --index example we'd have all the information we need for --skip-existing to remain a boolean flag. I think a change like that would require quite a bit of discussion though, and I don't want to derail improvements here, so my goal is to question if there's a way to do this that's consistent with other tooling and provides a path to a better experience in the long run?

Could we have --skip-existing work as as a boolean flag and provide an optional flag for an index URL (e.g., --query-url) to improve its behavior? It's not clear to me if --skip-existing should fail when used with non-PyPI indexes without an additional --query-url or just operate on a best-effort basis and provide a hint on failure.

As a closing note, I worry about optimizing the --skip-existing behavior for non-PyPI servers. I think if someone is using a non-standard server than it's reasonable to provide an additional flag.

konstin · 2024-10-29T09:09:00Z

with uv publish --index example we'd have all the information we need for --skip-existing to remain a boolean flag. I think a change like that would require quite a bit of discussion though, and I don't want to derail improvements here, so my goal is to question if there's a way to do this that's consistent with other tooling and provides a path to a better experience in the long run?

I like the uv publish --index example: It's concise, it has toml configuration and it pairs the publish URL with the index URL directly. We should provide both a named index option and individual options, so I'm fine with adding --index in a follow-up.

Could we have --skip-existing work as as a boolean flag and provide an optional flag for an index URL (e.g., --query-url) to improve its behavior? It's not clear to me if --skip-existing should fail when used with non-PyPI indexes without an additional --query-url or just operate on a best-effort basis and provide a hint on failure.

As a closing note, I worry about optimizing the --skip-existing behavior for non-PyPI servers. I think if someone is using a non-standard server than it's reasonable to provide an additional flag.

We should provide consistent publish experience no matter the server behavior. There is no guarantee a server behaves a certain way (or changes that way), and only an index URL gives us confidence when checking.

We can't consider non-PyPI servers non-standard, there is no standard (that would be the long-awaited Upload API 2.0), and I believe we shouldn't consider internal index users second class. All implementations exist in a back-and-forth between twine as client and the various servers: Twine itself contains special casing for their skip existing implementation for various servers (https://github.com/pypa/twine/blob/c512bbf166ac38239e58545a39155285f8747a7b/twine/commands/upload.py#L58-L72). An advantage of the skip existing with URL in that it means no more special casing heuristics.

The boolean skip existing option can't differentiate between matching and mismatching re-uploads, which is crucial to catch mismatching uploads, and it also doesn't allow skipping uploads of files that are already uploaded. Since users will need to know their index URL anyway for using the uploaded package, we can make this --query-url mandatory, which here is called --skip-existing.

zanieb · 2024-10-29T17:55:30Z

We can't consider non-PyPI servers non-standard, there is no standard (that would be the long-awaited Upload API 2.0), and I believe we shouldn't consider internal index users second class.

I think you're picking at the language here. I think we can consider servers that are not the official Python package index as unofficial. I don't think they need to be second-class, but the they're already secondary, e.g., uv publish uses PyPI by default.

Can you clarify the existing behaviors for me, i.e., for twine? A repeated upload with the same hash to PyPI results in an OK so --skip-existing isn't needed, that's just the default behavior? Other indexes do inconsistent things, i.e., erroring so --skip-existing needs to ignore special-cased error messages?

konstin · 2024-10-29T19:06:33Z

Skip existing makes twine ignore all error responses matching https://github.com/pypa/twine/blob/c512bbf166ac38239e58545a39155285f8747a7b/twine/commands/upload.py#L58-L72. See also #7917 (comment) for more cross index testing.

GitLab

Uploading the same file for the same name

twine upload --repository-url https://gitlab.com/api/v4/projects/61853105/packages/pypi scripts/publish/astral-test-token/dist/astral_test_token-0.1.730-py3-none-any.whl

Error 400 Bad Request

twine upload --repository-url --skip-existing https://gitlab.com/api/v4/projects/61853105/packages/pypi scripts/publish/astral-test-token/dist/astral_test_token-0.1.730-py3-none-any.whl

Ok

Uploading a different file for the same name

twine upload --repository-url https://gitlab.com/api/v4/projects/61853105/packages/pypi scripts/publish/astral-test-token-modified/dist/astral_test_token-0.1.730-py3-none-any.whl

Error 400 Bad Request

twine upload --repository-url --skip-existing https://gitlab.com/api/v4/projects/61853105/packages/pypi scripts/publish/astral-test-token-modified/dist/astral_test_token-0.1.730-py3-none-any.whl

Ok

PyPI

Using PyPI instead of GitLab gives a slightly different pattern:

Uploading the same file for the same name

twine upload -r testpypi scripts/publish/astral-test-token/dist/astral_test_token-0.1.730-py3-none-any.whl

OK

twine upload -r testpypi --skip-existing scripts/publish/astral-test-token/dist/astral_test_token-0.1.730-py3-none-any.whl

Ok

Uploading a different file for the same name

twine upload -r testpypi scripts/publish/astral-test-token-modified/dist/astral_test_token-0.1.730-py3-none-any.whl

Error 400 Bad Request

twine upload -r testpypi --skip-existing scripts/publish/astral-test-token-modified/dist/astral_test_token-0.1.730-py3-none-any.whl

Ok

konstin · 2024-10-29T23:18:35Z

The main design constraint here is that i feel strongly that the behavior should be consistent across all indexes, and I've picked the design with a mandatory index URL since it was the only one that could provide this consistency.

zanieb · 2024-10-29T23:29:20Z

So.. in a future world where we use --index instead of --publish-url, does --skip-existing become the default behavior? Is there a world in which you would not want to skip existing, identical objects? (Trying to inform a decision on the flag name)

konstin · 2024-10-29T23:41:02Z

Yes, i'm planning to add --index and make skip existing the default behavior with this; I couldn't come up with any case where we didn't want to skip existing, identical objects if we can.

crates/uv-publish/src/lib.rs

Co-authored-by: Zanie Blue <[email protected]>

crates/uv-publish/src/lib.rs

See #8531 (comment), we hint users coming from twine to use `--check-url` instead. > `uv publish` does not support `--skip-existing`, use `--check-url` with the simple index URL instead. --------- Co-authored-by: Zanie Blue <[email protected]>

Fixes astral-sh#9027 Minor enhancement on top of astral-sh#8531 that allows CLI parameter `--check-url` also available as the setting `check-url` in configuration files.

Fixes astral-sh#9027 Minor enhancement on top of astral-sh#8531 that makes the CLI parameter `--check-url` also available as the setting `check-url` in configuration files.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | minor | `0.4.24` -> `0.5.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.5.2`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#052) [Compare Source](astral-sh/uv@0.5.1...0.5.2) ##### Enhancements - Hide `--no-system` from `uv pip tree` CLI ([#9040](astral-sh/uv#9040)) - Allow configuration of Python and PyPy install mirrors in `uv.toml` ([#8695](astral-sh/uv#8695)) - Allow passing Python download mirrors to `uv python install` ([#8695](astral-sh/uv#8695)) - Add support for specifying conflicting extras and dependency groups ([#8976](astral-sh/uv#8976), [#9096](astral-sh/uv#9096)) - Consistent colon usage in build failure errors ([#8994](astral-sh/uv#8994)) - Show full derivation chain when encountering build failures ([#9108](astral-sh/uv#9108)) - Show link we failed on parsing index pages ([#9118](astral-sh/uv#9118)) - Remove duplicate log when searching for interpreters ([#9092](astral-sh/uv#9092)) - Update uv development status classifier to "Stable" on PyPI ([#8943](astral-sh/uv#8943)) - Use rich diagnostic formatting for early build failures ([#9041](astral-sh/uv#9041)) - Use rich diagnostic formatting for install failures ([#9043](astral-sh/uv#9043)) ##### Performance - Avoid retraversing filesystem when testing exact glob matches ([#9022](astral-sh/uv#9022)) ##### Bug fixes - Allow `--no-build` to validate lock ([#9024](astral-sh/uv#9024)) - Allow default indexes to be marked as explicit ([#8990](astral-sh/uv#8990)) - Avoid creating `.venv` in `uv add --frozen` and `uv add --no-sync` ([#8980](astral-sh/uv#8980)) - Avoid duplicating first-entry comments in `uv add` ([#9109](astral-sh/uv#9109)) - Defer reporting of build failures in resolver ([#9098](astral-sh/uv#9098)) - Fix references to `--resolution-strategy` in error message output ([#8971](astral-sh/uv#8971)) - Ignore virtual environments in parent directories when choosing Python version for new projects ([#9075](astral-sh/uv#9075)) - Forward SIGTERM to child processes in `uv run` ([#8933](astral-sh/uv#8933)) - Prefer Python executable names that match the request over default names ([#9066](astral-sh/uv#9066)) - Prefer compatible to incompatible distributions when packages exist on multiple indexes ([#8961](astral-sh/uv#8961)) - Publish: Ignore non-matching files ([#8986](astral-sh/uv#8986)) - Revert `uv.lock` changes when `uv add` fails ([#9030](astral-sh/uv#9030)) - Show file extensions on available commands when not `.exe` ([#9099](astral-sh/uv#9099)) - Sort by name, then specifiers in `uv add` ([#9097](astral-sh/uv#9097)) - Split after specifiers in `--with` requirements ([#9089](astral-sh/uv#9089)) - Support multiple extras in universal pip compile output ([#8960](astral-sh/uv#8960)) ##### Preview features - Build backend: Add tests for source tree -> source dist -> wheel conversions ([#9091](astral-sh/uv#9091)) - Build backend: Switch to custom `glob-walkdir` implementation ([#9013](astral-sh/uv#9013)) - Build backend: Add minimal wheel settings ([#9085](astral-sh/uv#9085)) ##### Documentation - Add wget instructions for systems without curl ([#8630](astral-sh/uv#8630)) - Fix `.env` file example in docs ([#9064](astral-sh/uv#9064)) - Fix reference to `--resolution` in docs ([#8968](astral-sh/uv#8968)) - Fix typo in GitLab integration docs ([#9047](astral-sh/uv#9047)) - Update format of environment variable reference ([#9018](astral-sh/uv#9018)) - Use Python syntax for `value_type` consistently ([#9017](astral-sh/uv#9017)) - Use `[[index]]` API in configuration example ([#9065](astral-sh/uv#9065)) - Mention how to use extras ([#8972](astral-sh/uv#8972)) - Add some words about specifying conflicting extras/groups ([#9120](astral-sh/uv#9120)) ### [`v0.5.1`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#051) [Compare Source](astral-sh/uv@0.5.0...0.5.1) ##### Enhancements - Allow installation of manylinux wheels on `riscv64` ([#8934](astral-sh/uv#8934)) ##### Bug fixes - Build source distributions at top-level of cache ([#8905](astral-sh/uv#8905)) - Allow non-registry dependencies in `uv pip list --outdated` ([#8939](astral-sh/uv#8939)) - Compute superset of existing and required hashes when healing cache ([#8955](astral-sh/uv#8955)) - Enable uv to replace and delete itself on Windows ([#8914](astral-sh/uv#8914)) - Remove source distribution filename from cache ([#8907](astral-sh/uv#8907)) - Respect `--index-url` in `uv pip list` ([#8942](astral-sh/uv#8942)) - Respect comma-separated extras in `--with` ([#8946](astral-sh/uv#8946)) ##### Documentation - Add uninstall note for previous versions ([#8937](astral-sh/uv#8937)) - Remove some missed references to `~/.cargo/bin` ([#8936](astral-sh/uv#8936)) - Split README's install code block into 3 ([#8853](astral-sh/uv#8853)) ### [`v0.5.0`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#050) [Compare Source](astral-sh/uv@0.4.30...0.5.0) Since the launch of Python version, project, and tool management capabilities back in August, we've seen extraordinary adoption of uv. We've been iterating rapidly: adding new features, fixing bugs, and improving the user experience. Despite moving quickly, stability and compatibility have been a major focus — we've made thirty releases since our last breaking change. Consequently, we've accumulated various changes that improve correctness and user experience, but could break some workflows. This release contains those changes; many have been marked as breaking out of an abundance of caution. We expect most users to be able to upgrade without making changes. ##### Breaking - **Use base executable to set virtualenv Python path** ([#8481](astral-sh/uv#8481)) Previously, uv canonicalized the path to the Python executable when setting the Python path in created virtual environments. This behavior had several undesirable effects: it would bypass stabilized version directories (as constructed by Homebrew) and it was not consistent with the Python standard library's behavior. Now, uv uses the `sys._base_executable` path. - **Use XDG (i.e. `~/.local/bin`) instead of the Cargo home directory in the installer** ([#8420](astral-sh/uv#8420)) Previously, uv's installer used `$CARGO_HOME` or `~/.cargo/bin` for its target install directory. It's been a longstanding complaint that uv uses this directory, as there's no relationship to Cargo. Now, uv will be installed into `$XDG_BIN_HOME`, `$XDG_DATA_HOME/../bin`, or `~/.local/bin` (in that order). Note that `$UV_INSTALL_DIR` can always be used to override the target directory. - **Discover and respect `.python-version` files in parent directories** ([#6370](astral-sh/uv#6370)) Previously, uv only read `.python-version` files from the working directory. Now, uv will check parent directories for `.python-version` files; however uv will not search for `.python-version` files beyond project boundaries. The new behavior is better aligned with that of `pyenv` and Rye. - **Error when disallowed settings are defined in `uv.toml`** ([#8550](astral-sh/uv#8550)) Some settings can only be defined in the `pyproject.toml`. Previously, uv would ignore these settings when present in the `uv.toml`. Now, uv will error to avoid confusion about why the settings are not respected. - **Implement PEP 440-compliant local version semantics** ([#8797](astral-sh/uv#8797)) Previously, uv's implementation of local versions (e.g. `2.0+cpu`) was not compliant with the specification due to the technical complexity of implementing the local version semantics in the PubGrub algorithm. Thanks to the work of [@ericmarkmartin](https://github.com/ericmarkmartin), uv now has a spec-compliant implementation. Namely, uv will now allow a request for `torch==2.1.0` to install `[email protected]+cpu` regardless of whether `[email protected]` (without a local tag) actually exists. - **Treat the base Conda environment as a system environment** ([#7691](astral-sh/uv#7691)) Previously, uv would not distinguish between the base and other Conda environments. Now, uv uses `CONDA_DEFAULT_ENV` and the names `base` and `default` to determine if an environment active via `CONDA_PREFIX` is the base environment. If the base environment is active, the `--system` flag must be used to mutate it. - **Do not allow pre-releases when the `!=` operator is used** ([#7974](astral-sh/uv#7974)) Previously, uv would use the presence of a pre-release specifier in a version specifier as an opt-in to allow pre-release versions during resolution. The new behavior does not allow pre-releases when an inequals operator is used, e.g., `!= 2.0a1`. - **Prefer `USERPROFILE` over `FOLDERID_Profile` when selecting a home directory on Windows** ([#8048](astral-sh/uv#8048)) This change is a side-effect of switching from the `directories` crate to `etcetera` for determining canonical system paths. If `USERPROFILE` is not set, the behavior will be unchanged. - **Improve interactions between color environment variables and CLI options** ([#8215](astral-sh/uv#8215)) Previously, uv would respect the `FORCE_COLOR` and `NO_COLOR` environment variables over the `--color` flag. Now, when the `--color` flag is explicitly provided, uv will respect it over the environment variables. - **Make `allow-insecure-host` a global option** ([#8476](astral-sh/uv#8476)) Previously, this option was only available in some parts of uv. Now, `--allow-insecure-host` can be provided to any command. For consistency, the `allow-insecure-host` setting has been removed from the `[tool.uv.pip]` configuration in favor of `[tool.uv]`. - **Only write `.python-version` files during `uv init` for workspace members if the version differs** ([#8897](astral-sh/uv#8897)) Previously, uv would create a `.python-version` file for workspace members during `uv init`. Now, uv will only do so if the version differs from the `.python-version` file in the workspace root since uv will respect `.python-version` files in parent directories. ##### Enhancements - Add `uv tree --outdated` ([#8893](astral-sh/uv#8893)) - Add armv8l alias for armv7l to support arm 32-bit compatibility mode ([#8881](astral-sh/uv#8881)) - Add support for `pip list --outdated` ([#8872](astral-sh/uv#8872)) - Allow semicolons directly after direct URLs ([#8836](astral-sh/uv#8836)) - Enable support for arbitrary git transports ([#8769](astral-sh/uv#8769)) - Improve Python discovery source messages ([#8890](astral-sh/uv#8890)) - Show dedicated error for trailing `;` on URL and path requirements ([#8835](astral-sh/uv#8835)) - Add progress bar for `uv cache clean` ([#8857](astral-sh/uv#8857)) - Warn on failure to query system configuration file ([#8829](astral-sh/uv#8829)) ##### Preview features - Add support for building basic source distributions with the experimental uv build backend ([#8886](astral-sh/uv#8886)) ##### Bug fixes - Respect dynamic version updates in `uv lock` ([#8867](astral-sh/uv#8867)) - Respect fork markers in `--resolution-mode=lowest-direct` ([#8839](astral-sh/uv#8839)) ##### Documentation - Add further examples of git+https support ([#8841](astral-sh/uv#8841)) - Add installer variables to environment reference ([#8874](astral-sh/uv#8874)) - Add note on private classifier ([#8783](astral-sh/uv#8783)) - Update pip-and-uv strictness example ([#8822](astral-sh/uv#8822)) - Fix `uv python install` docs to use an existing PyPy version ([#8845](astral-sh/uv#8845)) - Document how to mimic `--verbose` with `RUST_LOG` ([#8858](astral-sh/uv#8858)) ### [`v0.4.30`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0430) [Compare Source](astral-sh/uv@0.4.29...0.4.30) ##### Enhancements - Add support for `.env` and custom env files in `uv run` ([#8811](astral-sh/uv#8811)) - Add support for `--all-packages` in `uv run`, `uv sync`, and `uv export` ([#8742](astral-sh/uv#8742), [#8741](astral-sh/uv#8741), [#8739](astral-sh/uv#8739)) - Allow use of `--frozen` with `--all-packages` in `uv sync` and `uv export` ([#8760](astral-sh/uv#8760)) - Show full error chain on tool upgrade failures ([#8753](astral-sh/uv#8753)) - Add `--check-url` to `uv publish` to check for existing distributions during upload ([#8531](astral-sh/uv#8531)) - Suggest using `--check-url` when `--skip-existing` is used ([#8803](astral-sh/uv#8803)) ##### Bug fixes - Allow incompatible `requires-python` for source distributions with static metadata ([#8768](astral-sh/uv#8768)) - Allow managed downloads with `--python-preference system` ([#8808](astral-sh/uv#8808)) - Avoid error for `--group` defined in non-root workspace member ([#8734](astral-sh/uv#8734)) - Avoid showing dependency group annotations on workspace members in tree ([#8730](astral-sh/uv#8730)) - Do not error when the Python bin directory is missing on `uv python uninstall` ([#8725](astral-sh/uv#8725)) - Include member groups when locking workspace ([#8736](astral-sh/uv#8736)) - Fix bug where `python_version < '0'` could appear in a final resolution ([#8759](astral-sh/uv#8759)) - Sanitize filenames during zip extraction ([#8732](astral-sh/uv#8732)) - Switch to RFC 9110 compatible format for exclude newer requests ([#8752](astral-sh/uv#8752)) ##### Preview features - Add support for installing versioned Python executables on Windows ([#8663](astral-sh/uv#8663)) - Improve interactions with existing Python executables during install ([#8733](astral-sh/uv#8733)) ##### Rust API - Extend `BaseClient` to accept extra middleware ([#8807](astral-sh/uv#8807)) - Add `From` for `FlatDistributions` struct ([#8800](astral-sh/uv#8800)) ##### Documentation - Fix environment variable name in providing credentials section ([#8740](astral-sh/uv#8740)) - Fix `add httpx` example with real git branch ([#8756](astral-sh/uv#8756)) - Fix indentation in `projects.md` ([#8772](astral-sh/uv#8772)) - Fix link to publish guide in `README` ([#8720](astral-sh/uv#8720)) - Generate environment variables documentation from code ([#8493](astral-sh/uv#8493)) - Improve and fix some documents ([#8749](astral-sh/uv#8749)) - Improve environment variables document ([#8777](astral-sh/uv#8777)) ### [`v0.4.29`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0429) [Compare Source](astral-sh/uv@0.4.28...0.4.29) ##### Enhancements - Sort errors during display in `uv python install` ([#8684](astral-sh/uv#8684)) - Update resolver to use disjointness checks instead of marker equality ([#8661](astral-sh/uv#8661)) - Add `riscv64` to supported Python platform tags ([#8660](astral-sh/uv#8660)) ##### Bug fixes - Fix hard and soft float libc detection for managed Python distributions on ARM ([#8498](astral-sh/uv#8498)) - Handle cycles in `uv pip tree` ([#8689](astral-sh/uv#8689)) - Respect dependency group markers in `uv export` ([#8659](astral-sh/uv#8659)) - Support transitive dependencies in Git workspaces ([#8665](astral-sh/uv#8665)) - Use portable paths for subdirectories in lock URLs ([#8707](astral-sh/uv#8707)) - Update `uv init --virtual` to imply `--no-package` ([#8595](astral-sh/uv#8595)) ##### Preview - Install versioned Python executables into the bin directory during `uv python install` (Unix only) ([#8458](astral-sh/uv#8458)) ##### Documentation - Clarify relationship between specifiers and `requires-python` range ([#8688](astral-sh/uv#8688)) - Fix broken link in docs ([#8552](astral-sh/uv#8552)) - Fix outdated documentation on `Requires-Python` ([#8679](astral-sh/uv#8679)) - Add Google Artifact Registry index authentication guide ([#8579](astral-sh/uv#8579)) ### [`v0.4.28`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0428) [Compare Source](astral-sh/uv@0.4.27...0.4.28) ##### Enhancements - Add support for requesting free-threaded builds via `+freethreaded` ([#8645](astral-sh/uv#8645)) - Improve trusted publishing error messages ([#8633](astral-sh/uv#8633)) - Remove unneeded `return` from Maturin project template ([#8604](astral-sh/uv#8604)) - Skip Python interpreter discovery for `uv export` ([#8638](astral-sh/uv#8638)) - Hint about missing trusted publishing permission ([#8632](astral-sh/uv#8632)) ##### Configuration - Add environment variable to disable progress output ([#8600](astral-sh/uv#8600)) ##### Bug fixes - Fork when minimum Python version increases ([#8628](astral-sh/uv#8628)) - Ignore empty groups when validating lock ([#8598](astral-sh/uv#8598)) - Remove duplicate word in error message ([#8589](astral-sh/uv#8589)) - Support cyclic dependencies in `uv tree` ([#8564](astral-sh/uv#8564)) - Update `uv init` to imply `--package` when using `--build-backend` ([#8593](astral-sh/uv#8593)) - Restore use of `dev-dependencies` and `requires-dev` for lockfile compatibility ([#8599](astral-sh/uv#8599)) ##### Documentation - Clarify `requires-python` requirement for dependencies ([#8619](astral-sh/uv#8619)) - Update CLI documentation for `--cache-dir` ([#8627](astral-sh/uv#8627)) ### [`v0.4.27`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0427) [Compare Source](astral-sh/uv@0.4.26...0.4.27) This release includes support for the `[dependency-groups]` table as recently standardized in [PEP 735](https://peps.python.org/pep-0735/). The table allows for declaration of optional dependency groups that are not published as part of the package metadata, unlike `[project.optional-dependencies]`. There are new `--group`, `--only-group`, and `--no-group` options throughout the uv interface. Previously, uv used a single `tool.uv.dev-dependencies` list for declaration of development dependencies. Now, uv supports declaring development dependencies in a standardized format and allows splitting development dependencies into multiple groups. For compatibility, and to simplify usage for people that do not need multiple groups, uv special-cases the group named `dev`. The `dev` group is equivalent to `tool.uv.dev-dependencies`. The contents of `tool.uv.dev-dependencies` will merged into the `dev` group in uv's resolver. The `--dev`, `--only-dev`, and `--no-dev` flags remain as aliases for the corresponding `--group` options. Support for `tool.uv.dev-dependencies` remains in this release, but will display warnings in a future release. uv syncs the `dev` group by default — this matches the exististing behavior for `tool.uv.dev-dependencies`. The default groups can be changed with the `tool.uv.default-groups` setting. Thank you to Stephen Rosen who authored PEP 735. ##### Enhancements - Support for PEP 735 ([#8272](astral-sh/uv#8272)) - Add support for `--dry-run` mode in `uv lock` ([#7783](astral-sh/uv#7783)) - Don't allow non-string email in authors ([#8520](astral-sh/uv#8520)) - Enforce lockfile schema versions ([#8509](astral-sh/uv#8509)) ##### Bug fixes - Always attach URL to network errors ([#8444](astral-sh/uv#8444)) - Fix dangling non-platform dependencies in `uv tree` ([#8532](astral-sh/uv#8532)) - Prefer `lto` over `debug` free-threaded managed Python builds ([#8515](astral-sh/uv#8515)) ##### Documentation - Add `tool.uv.sources` to the "Settings" reference ([#8543](astral-sh/uv#8543)) - Add reference to `uv build` and `uv publish` in the landing pages ([#8542](astral-sh/uv#8542)) - Avoid duplicate `[tool.uv]` header in TOML examples ([#8545](astral-sh/uv#8545)) - Document `.netrc` environment variable and path ([#8511](astral-sh/uv#8511)) - Fix `.netrc` typo in authentication docs ([#8521](astral-sh/uv#8521)) - Fix heading level of "Script support" on docs landing page ([#8544](astral-sh/uv#8544)) - Move the installation configuration docs to a separate page ([#8546](astral-sh/uv#8546)) - Update docs for `--publish-url` to avoid duplication. ([#8561](astral-sh/uv#8561)) - Fix typo ([#8554](astral-sh/uv#8554)) - Fix typo in description of `--strict` flag ([#8513](astral-sh/uv#8513)) ### [`v0.4.26`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0426) [Compare Source](astral-sh/uv@0.4.25...0.4.26) ##### Enhancements - Allow static dependency metadata entries for direct URL requirements ([#7846](astral-sh/uv#7846)) - Use reinstall report formatting for `uv python install --reinstall` ([#8487](astral-sh/uv#8487)) - Add support for system-level `uv.toml` configuration ([#7851](astral-sh/uv#7851)) ##### Bug fixes - Apply `requires-python` narrowing with upper bounds ([#8403](astral-sh/uv#8403)) - Avoid rewriting `[[tool.uv.index]]` entries when credentials are provided ([#8502](astral-sh/uv#8502)) - Fix `uv add` comment handling for empty arrays ([#8504](astral-sh/uv#8504)) - Replace dashes with underscores in index credential variables ([#8452](astral-sh/uv#8452)) - Respect `--allow-insecure-host` in `uv publish` ([#8440](astral-sh/uv#8440)) - Allow arbitrary `--package` includes in `uv tree` ([#8507](astral-sh/uv#8507)) - Remove existing Python install after successful download in `uv python install` ([#8485](astral-sh/uv#8485)) ##### Documentation - Add docs example for URLs with `[tool.uv.dependency-metadata]` ([#8484](astral-sh/uv#8484)) - Add help page for build failures ([#8286](astral-sh/uv#8286)) - Fix `cache-keys` typo in `tags = true` ([#8422](astral-sh/uv#8422)) - Add documentation examples for manual branch, rev, and tag Git dependencies ([#8497](astral-sh/uv#8497)) ##### Error messages - Improve error message for cache info serialization ([#8500](astral-sh/uv#8500)) - Suggest `--from` command when executable is available for `uvx` ([#8473](astral-sh/uv#8473)) - Support `--with-editable` in `uv tool install` ([#8472](astral-sh/uv#8472)) ### [`v0.4.25`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0425) [Compare Source](astral-sh/uv@0.4.24...0.4.25) ##### Enhancements - Add support for `uv pip show --files` ([#8369](astral-sh/uv#8369)) - Don't prefetch unreachable packages ([#8246](astral-sh/uv#8246)) - Remove `tool.uv.sources` table if it is empty ([#8365](astral-sh/uv#8365)) - Modify cache versioning to support backwards compatibility ([#8386](astral-sh/uv#8386)) ##### Configuration - Add support for `UV_FROZEN` and `UV_LOCKED` ([#8340](astral-sh/uv#8340)) ##### Bug fixes - Allow dashes and underscores in custom index names ([#8339](astral-sh/uv#8339)) - Avoid panic when Git dependencies are included in fork markers ([#8388](astral-sh/uv#8388)) - Check existing source by normalized name before `uv add` and `uv remove` ([#8359](astral-sh/uv#8359)) - Fix bug where username from authentication cache could be ignored ([#8345](astral-sh/uv#8345)) - Fix to respect comments positioning in pyproject.toml on change ([#8384](astral-sh/uv#8384)) - Redact index sources in `uv.lock` ([#8333](astral-sh/uv#8333)) - Use correct indentation when project table contains open bracket comment ([#8387](astral-sh/uv#8387)) - Only remove a source from `[tool.uv.sources]` if it is no long being referenced ([#8366](astral-sh/uv#8366)) - Modify `uv pip list` and `uv tree` to print to stdout regardless of `--quiet` flag ([#8392](astral-sh/uv#8392)) ##### Error messages - Improve help message for missing `self update` invocations ([#8337](astral-sh/uv#8337)) - Log `.netrc` parsing errors ([#8364](astral-sh/uv#8364)) - Remove trailing newlines in error messages ([#8322](astral-sh/uv#8322)) - Use a dedicated message for incompatible Python versions in wheel ABI tags ([#8363](astral-sh/uv#8363)) - Remove commands available in the top-level from the suggested subcommand error ([#8316](astral-sh/uv#8316)) ##### Release - Run release builds for `macos-x86_64` on `macos-14` runners ([#8327](astral-sh/uv#8327)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Minor enhancement on top of astral-sh#8531 that makes the CLI parameter `--check-url` also available as the setting `check-url` in configuration files.

## Summary Fixes #9027 Minor enhancement on top of #8531 that makes the CLI parameter `--check-url` also available as the setting `check-url` in configuration files. ## Test Plan Updates existing tests to take the new setting into account. Within publish command testing I didn't see existing tests covering settings from toml files (instead of from CLI params), so I didn't add anything of that sort.

konstin added enhancement New feature or improvement to existing functionality preview Experimental behavior labels Oct 24, 2024

konstin commented Oct 24, 2024

View reviewed changes

crates/uv-client/src/registry_client.rs Outdated Show resolved Hide resolved

konstin commented Oct 24, 2024

View reviewed changes

konstin had a problem deploying to uv-test-publish October 24, 2024 16:05 — with GitHub Actions Failure

konstin temporarily deployed to uv-test-publish October 24, 2024 16:22 — with GitHub Actions Inactive

konstin force-pushed the konsti/implement-skip-existing branch from 680abe5 to 656d4f0 Compare October 25, 2024 07:46

konstin had a problem deploying to uv-test-publish October 25, 2024 07:50 — with GitHub Actions Failure

konstin force-pushed the konsti/implement-skip-existing branch from 656d4f0 to 4c341a4 Compare October 28, 2024 09:27

konstin requested a review from charliermarsh October 28, 2024 09:27

konstin temporarily deployed to uv-test-publish October 28, 2024 09:31 — with GitHub Actions Inactive

charliermarsh reviewed Oct 28, 2024

View reviewed changes

zanieb reviewed Oct 28, 2024

View reviewed changes

crates/uv-cli/src/lib.rs Outdated Show resolved Hide resolved

zanieb reviewed Oct 28, 2024

View reviewed changes

crates/uv-cli/src/lib.rs Outdated Show resolved Hide resolved

konstin temporarily deployed to uv-test-publish October 28, 2024 14:51 — with GitHub Actions Inactive

konstin force-pushed the konsti/implement-skip-existing branch from 68c8ee8 to 1175fdf Compare October 29, 2024 08:36

konstin temporarily deployed to uv-test-publish October 29, 2024 08:40 — with GitHub Actions Inactive

konstin force-pushed the konsti/implement-skip-existing branch from 1175fdf to ac170f2 Compare October 29, 2024 19:06

konstin temporarily deployed to uv-test-publish October 29, 2024 19:10 — with GitHub Actions Inactive

konstin force-pushed the konsti/implement-skip-existing branch from ac170f2 to 0e409cc Compare October 29, 2024 23:21

konstin temporarily deployed to uv-test-publish October 29, 2024 23:27 — with GitHub Actions Inactive

zanieb reviewed Oct 31, 2024

View reviewed changes

crates/uv-publish/src/lib.rs Outdated Show resolved Hide resolved

Update crates/uv-cli/src/lib.rs

a56a228

Co-authored-by: Zanie Blue <[email protected]>

konstin temporarily deployed to uv-test-publish October 31, 2024 13:43 — with GitHub Actions Inactive

charliermarsh reviewed Oct 31, 2024

View reviewed changes

crates/uv-publish/src/lib.rs Outdated Show resolved Hide resolved

Avoid changing the registry client for refresh

a402013

konstin temporarily deployed to uv-test-publish October 31, 2024 14:17 — with GitHub Actions Inactive

Docstring

43593a0

konstin temporarily deployed to uv-test-publish October 31, 2024 14:25 — with GitHub Actions Inactive

Don't pass option to function

560390f

konstin temporarily deployed to uv-test-publish October 31, 2024 14:31 — with GitHub Actions Inactive

konstin merged commit 0822594 into main Oct 31, 2024
62 of 63 checks passed

konstin deleted the konsti/implement-skip-existing branch October 31, 2024 15:23

konstin mentioned this pull request Nov 3, 2024

Promote uv publish from preview to stable #7839

Open

19 tasks

cthoyt mentioned this pull request Nov 4, 2024

Configuring multiple indexes with uv publish #7963

Open

konstin mentioned this pull request Nov 4, 2024

Publish: Hint at --skip-existing -> --check-url transition #8803

Merged

BrewTestBot mentioned this pull request Nov 5, 2024

uv 0.4.30 Homebrew/homebrew-core#196679

Merged

This was referenced Nov 6, 2024

Support index declarations in uv publish #8864

Closed

uv publish trusted publish integration test fails with 400 (already exists) #8817

Closed

bruckner mentioned this pull request Nov 11, 2024

Ability to set --check-url option in pyproject.toml or uv.toml #9027

Closed

bruckner mentioned this pull request Nov 11, 2024

Make check-url available in configuration files #9032

Merged

charliermarsh mentioned this pull request Nov 12, 2024

[feat] uv publish --skip-existing #7917

Closed

bruckner added a commit to bruckner/uv that referenced this pull request Dec 1, 2024

Make check-url available in configuration files

e14ed9a

Minor enhancement on top of astral-sh#8531 that makes the CLI parameter `--check-url` also available as the setting `check-url` in configuration files.

bruckner added a commit to bruckner/uv that referenced this pull request Dec 2, 2024

Make check-url available in configuration files

4d8108d

Minor enhancement on top of astral-sh#8531 that makes the CLI parameter `--check-url` also available as the setting `check-url` in configuration files.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Skip existing, second iteration: Check the index before uploading #8531

Skip existing, second iteration: Check the index before uploading #8531

konstin commented Oct 24, 2024 •

edited

Loading

konstin Oct 24, 2024

konstin Oct 24, 2024

charliermarsh Oct 28, 2024

konstin Oct 24, 2024

charliermarsh Oct 28, 2024

konstin Oct 28, 2024

charliermarsh Oct 28, 2024

konstin Oct 28, 2024

konstin Oct 28, 2024

zanieb Oct 28, 2024

konstin Oct 28, 2024

zanieb commented Oct 28, 2024

konstin commented Oct 29, 2024

zanieb commented Oct 29, 2024

konstin commented Oct 29, 2024 •

edited

Loading

konstin commented Oct 29, 2024

zanieb commented Oct 29, 2024 •

edited

Loading

konstin commented Oct 29, 2024

Skip existing, second iteration: Check the index before uploading #8531

Skip existing, second iteration: Check the index before uploading #8531

Conversation

konstin commented Oct 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zanieb commented Oct 28, 2024

konstin commented Oct 29, 2024

zanieb commented Oct 29, 2024

konstin commented Oct 29, 2024 • edited Loading

GitLab

PyPI

konstin commented Oct 29, 2024

zanieb commented Oct 29, 2024 • edited Loading

konstin commented Oct 29, 2024

konstin commented Oct 24, 2024 •

edited

Loading

konstin commented Oct 29, 2024 •

edited

Loading

zanieb commented Oct 29, 2024 •

edited

Loading