Skip to content

Commit

Permalink
Don't prefetch unreachable packages
Browse files Browse the repository at this point in the history
When batch prefetching we can fetch versions we know that are incompatible. In the following example, we were prefetching sentry-kafka-schemas below version 1.50.0.

```
python-rapidjson<=1.20,>=1.4
sentry-kafka-schemas<=0.1.113,>=0.1.50
```

Using a new pubgrub interface from astral-sh/pubgrub#32, we can avoid those prefetches by asking for incompatibilities that won't change anymore (those with root).
  • Loading branch information
konstin committed Oct 17, 2024
1 parent d930367 commit 7475e75
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 4 deletions.
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ pathdiff = { version = "0.2.1" }
petgraph = { version = "0.6.5" }
platform-info = { version = "2.0.3" }
proc-macro2 = { version = "1.0.86" }
pubgrub = { git = "https://github.com/astral-sh/pubgrub", rev = "19c77268c0ad5f69d7e12126e0cfacfbba466481" }
pubgrub = { git = "https://github.com/astral-sh/pubgrub", rev = "7243f4faf8e54837aa8a401a18406e7173de4ad5" }
quote = { version = "1.0.37" }
rayon = { version = "1.10.0" }
reflink-copy = { version = "0.1.19" }
Expand Down
16 changes: 14 additions & 2 deletions crates/uv-resolver/src/resolver/batch_prefetch.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use std::cmp::min;

use itertools::Itertools;
use pubgrub::Range;
use pubgrub::{Range, Term};
use rustc_hash::FxHashMap;
use tokio::sync::mpsc::Sender;
use tracing::{debug, trace};
Expand Down Expand Up @@ -49,6 +49,7 @@ impl BatchPrefetcher {
index: Option<&IndexUrl>,
version: &Version,
current_range: &Range<Version>,
unchangeable_constraints: Option<&Term<Range<Version>>>,
python_requirement: &PythonRequirement,
request_sink: &Sender<Request>,
in_memory: &InMemoryIndex,
Expand Down Expand Up @@ -119,11 +120,22 @@ impl BatchPrefetcher {
}
}
BatchPrefetchStrategy::InOrder { previous } => {
let range = if selector.use_highest_version(name) {
let mut range = if selector.use_highest_version(name) {
Range::strictly_lower_than(previous)
} else {
Range::strictly_higher_than(previous)
};
// If we have constraints from root, don't go beyond those. Example: We are
// prefetching for foo 1.60 and have a dependency for `foo>=1.50`, so we should
// only prefetch 1.60 to 1.50, knowing 1.49 will always be rejected.
if let Some(unchangeable_constraints) = unchangeable_constraints {
range = match unchangeable_constraints {
Term::Positive(constraints) => range.intersection(constraints),
Term::Negative(negative_constraints) => {
range.intersection(&negative_constraints.complement())
}
};
}
if let Some(candidate) =
selector.select_no_preference(name, &range, version_map, markers)
{
Expand Down
4 changes: 4 additions & 0 deletions crates/uv-resolver/src/resolver/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,10 @@ impl<InstalledPackages: InstalledPackagesProvider> ResolverState<InstalledPackag
index,
&version,
term_intersection.unwrap_positive(),
state
.pubgrub
.partial_solution
.unchanging_term_for_package(&state.next),
&state.python_requirement,
&request_sink,
&self.index,
Expand Down

0 comments on commit 7475e75

Please sign in to comment.