cargo update downgrading crates, it is not clear why

[kamulos]

Problem

So I have a strange interaction between two tools. I am using cargo upgrade --incompatible from cargo-edit to update all packages even if the updates are semver incompatible. After that everything still works fine. Cargo builds without complaining, everything seems fine. But if I run cargo update after that cargo will downgrade some of the packages.

There might be a good reason for this, I mean cargo-edit is not an official tool and might do things in a wrong way. But it also is absolutely unclear why cargo downgrades some dependencies, especially since it is building without an issue before the downgrade.

Steps

1. clone https://github.com/kamulos/cargoissue
2. run cargo upgrade --incompatible (version 0.13.0) => a few packages will be updated and the Cargo.toml will be changed
3. run cargo update the following output will appear

    Updating crates.io index
     Locking 4 packages to latest compatible versions
 Downgrading litemap v0.7.4 -> v0.7.3 (available: v0.7.4)
    Updating ureq v2.10.1 -> v2.11.0
 Downgrading yoke v0.7.5 -> v0.7.4 (available: v0.7.5)
 Downgrading zerofrom v0.1.5 -> v0.1.4 (available: v0.1.5)

Possible Solution(s)

No response

Notes

No response

Version

cargo 1.85.0-nightly (4c39aaff6 2024-11-25)
release: 1.85.0-nightly
commit-hash: 4c39aaff66862cc0da52fe529aa1990bb8bb9a22
commit-date: 2024-11-25
host: x86_64-unknown-linux-gnu
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.9.0-DEV (sys:0.4.74+curl-8.9.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Arch Linux [64-bit]

[epage]

cargo upgrade is not needed to reproduce this.

With where versions are right now, cargo upgrade is just updating version requirements to match the lockfile.

Just apply

diff --git c/Cargo.toml w/Cargo.toml
index 0ac6a10..f6aec1f 100644
--- c/Cargo.toml
+++ w/Cargo.toml
@@ -4,5 +4,5 @@ edition = "2021"
 publish = false

 [dependencies]
-libsodium-sys-stable = "1.21.1"
-url = "2.5.2"
+libsodium-sys-stable = "1.22.1"
+url = "2.5.4"

Before that patch:

$ cargo update
warning: ignoring `resolver` config table without `-Zmsrv-policy`
    Updating crates.io index
     Locking 6 packages to latest compatible versions
    Updating allocator-api2 v0.2.20 -> v0.2.21
    Updating cc v1.2.1 -> v1.2.2
    Updating errno v0.3.9 -> v0.3.10
    Updating indexmap v2.6.0 -> v2.7.0
    Updating libc v0.2.166 -> v0.2.167
    Updating syn v2.0.89 -> v2.0.90
note: pass `--verbose` to see 1 unchanged dependencies behind latest

With that patch

$ cargo update
warning: ignoring `resolver` config table without `-Zmsrv-policy`
    Updating crates.io index
     Locking 10 packages to latest compatible versions
    Updating allocator-api2 v0.2.20 -> v0.2.21
    Updating cc v1.2.1 -> v1.2.2
    Updating errno v0.3.9 -> v0.3.10
    Updating indexmap v2.6.0 -> v2.7.0
    Updating libc v0.2.166 -> v0.2.167
 Downgrading litemap v0.7.4 -> v0.7.3 (available: v0.7.4)
    Updating syn v2.0.89 -> v2.0.90
    Updating ureq v2.10.1 -> v2.11.0
 Downgrading yoke v0.7.5 -> v0.7.4 (available: v0.7.5)
 Downgrading zerofrom v0.1.5 -> v0.1.4 (available: v0.1.5)

Its interesting that ureq is deciding to upgrade as well as the downgrades

Looking at how those get pulled in

$ cargo tree -i litemap
warning: ignoring `resolver` config table without `-Zmsrv-policy`
litemap v0.7.3
├── icu_locid v1.5.0
│   ├── icu_locid_transform v1.5.0
│   │   └── icu_properties v1.5.1
│   │       ├── icu_normalizer v1.5.0
│   │       │   └── idna_adapter v1.2.0
│   │       │       └── idna v1.0.3
│   │       │           └── url v2.5.4
│   │       │               ├── cargobug v0.0.0 (/home/epage/src/personal/dump/cargoissue)
│   │       │               └── ureq v2.11.0
│   │       │                   [build-dependencies]
│   │       │                   └── libsodium-sys-stable v1.22.1
│   │       │                       └── cargobug v0.0.0 (/home/epage/src/personal/dump/cargoissue)
│   │       └── idna_adapter v1.2.0 (*)
│   └── icu_provider v1.5.0
│       ├── icu_locid_transform v1.5.0 (*)
│       ├── icu_normalizer v1.5.0 (*)
│       └── icu_properties v1.5.1 (*)
└── ureq v2.11.0 (*)

$ cargo tree -i yoke
warning: ignoring `resolver` config table without `-Zmsrv-policy`
  Downloaded errno v0.3.10
  Downloaded allocator-api2 v0.2.21
  Downloaded indexmap v2.7.0
  Downloaded cc v1.2.2
  Downloaded ureq v2.11.0
  Downloaded syn v2.0.90
  Downloaded libc v0.2.167
  Downloaded 7 crates (1.4 MB) in 0.72s
yoke v0.7.4
├── icu_collections v1.5.0
│   ├── icu_normalizer v1.5.0
│   │   └── idna_adapter v1.2.0
│   │       └── idna v1.0.3
│   │           └── url v2.5.4
│   │               ├── cargobug v0.0.0 (/home/epage/src/personal/dump/cargoissue)
│   │               └── ureq v2.11.0
│   │                   [build-dependencies]
│   │                   └── libsodium-sys-stable v1.22.1
│   │                       └── cargobug v0.0.0 (/home/epage/src/personal/dump/cargoissue)
│   └── icu_properties v1.5.1
│       ├── icu_normalizer v1.5.0 (*)
│       └── idna_adapter v1.2.0 (*)
├── icu_provider v1.5.0
│   ├── icu_locid_transform v1.5.0
│   │   └── icu_properties v1.5.1 (*)
│   ├── icu_normalizer v1.5.0 (*)
│   └── icu_properties v1.5.1 (*)
├── ureq v2.11.0 (*)
└── zerovec v0.10.4
    ├── icu_collections v1.5.0 (*)
    ├── icu_locid v1.5.0
    │   ├── icu_locid_transform v1.5.0 (*)
    │   └── icu_provider v1.5.0 (*)
    ├── icu_locid_transform v1.5.0 (*)
    ├── icu_normalizer v1.5.0 (*)
    ├── icu_properties v1.5.1 (*)
    ├── icu_provider v1.5.0 (*)
    └── tinystr v0.7.6
        ├── icu_locid v1.5.0 (*)
        ├── icu_locid_transform v1.5.0 (*)
        ├── icu_properties v1.5.1 (*)
        └── icu_provider v1.5.0 (*)

$ cargo tree -i zerofrom
warning: ignoring `resolver` config table without `-Zmsrv-policy`
zerofrom v0.1.4
├── icu_collections v1.5.0
│   ├── icu_normalizer v1.5.0
│   │   └── idna_adapter v1.2.0
│   │       └── idna v1.0.3
│   │           └── url v2.5.4
│   │               ├── cargobug v0.0.0 (/home/epage/src/personal/dump/cargoissue)
│   │               └── ureq v2.11.0
│   │                   [build-dependencies]
│   │                   └── libsodium-sys-stable v1.22.1
│   │                       └── cargobug v0.0.0 (/home/epage/src/personal/dump/cargoissue)
│   └── icu_properties v1.5.1
│       ├── icu_normalizer v1.5.0 (*)
│       └── idna_adapter v1.2.0 (*)
├── icu_provider v1.5.0
│   ├── icu_locid_transform v1.5.0
│   │   └── icu_properties v1.5.1 (*)
│   ├── icu_normalizer v1.5.0 (*)
│   └── icu_properties v1.5.1 (*)
├── ureq v2.11.0 (*)
├── yoke v0.7.4
│   ├── icu_collections v1.5.0 (*)
│   ├── icu_provider v1.5.0 (*)
│   ├── ureq v2.11.0 (*)
│   └── zerovec v0.10.4
│       ├── icu_collections v1.5.0 (*)
│       ├── icu_locid v1.5.0
│       │   ├── icu_locid_transform v1.5.0 (*)
│       │   └── icu_provider v1.5.0 (*)
│       ├── icu_locid_transform v1.5.0 (*)
│       ├── icu_normalizer v1.5.0 (*)
│       ├── icu_properties v1.5.1 (*)
│       ├── icu_provider v1.5.0 (*)
│       └── tinystr v0.7.6
│           ├── icu_locid v1.5.0 (*)
│           ├── icu_locid_transform v1.5.0 (*)
│           ├── icu_properties v1.5.1 (*)
│           └── icu_provider v1.5.0 (*)
└── zerovec v0.10.4 (*)

Looking at ureq

# These are locked down for MSRV 1.67
rustls = { version = "=0.23.19", optional = true, default-features = false, features = ["ring", "logging", "std", "tls12"] }
yoke = "=0.7.4"
litemap = "=0.7.3"
zerofrom = "=0.1.4"

These weren't locked down in 2.10.1, see https://github.com/algesten/ureq/blob/2.10.1/Cargo.toml

[epage]

My best guess as to what is happening is this is dependent on the order things get resolved

• If yoke, litemap, or zerofrom are picked first, then the older ureq is picked
• If ureq is picked first, then the older yoke, litemap, and zerofrom are picked

However, I'm not see why changing to compatible version requirements would change the walk order of the dependency tree. @Eh2406 any thoughts?

Besides the lack of consistency in our walk order, I would consider this a bug within ureq. They are trying to enforce MSRV through version reqs which we discourage as it can cause ecosystem-wide pain as has been shown repeatedly. This also prevents their dependents with higher MSRVs from using newer versions. MSRVs should instead be handled through lockfiles. This will be improved in 1.84 as we'll have an MSRV-aware resolver.

[kamulos]

Ah the dependency specification of rustls = "=0.23.19" sounds really bad 😕 Just recently I wanted to update rustls because it had a CVE, but cargo update -p rustls was doing nothing and telling me nothing why this would not work.

So maybe a feature request on the side would be to make issues like that discoverable. Maybe cargo could explain this while updating or maybe cargo tree could show which version requirements each crate sets.

[epage]

Made a note about this at algesten/ureq#878

[epage]

#7929 is our issue for explaining why upgrades do not happen.

[Eh2406]

Which lock file cargo generates is not guaranteed. We only guarantee that we try package versions from highest to lowest. We do not guarantee the order we try packages. In situations like this, there are more than one valid lock file that meets that requirement. In practice it is based on a number of implementation details, including how are priority queue deals with duplicates (which could change on any time cargo updates its dependencies), and the order that cargo provides the dependencies to the resolver, and probably many more I haven't thought about. While we do not guarantee any consistency here, we attempt to maintain it, because otherwise debugging the resolver is nearly impossible. All of those implementation details could affect resolution, but do not change from one run of cargo to the next. The relevant heuristic here is that we try requirements with fewer candidates first. We do this because in practice it leads to an exponential speed up in resolution time. Updating a minimum requiremment, even if the version chosen does not change, will reduce the number of candidates that could match. Meaning that it will be decided on earlier, and its dependencies will be provided to the resolver earlier. In this case by loading libsodium-sys-stable earlier we end up resolving ureq earlier. Presumably, before we had one of the three that changed being resolved before ureq.