Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add better diagnostics for rebuilding
This commit overhauls how a `Fingerprint` is stored on the filesystem and in-memory to help provide much better diagnostics as to why crates are being rebuilt. This involves storing more structured data on the filesystem in order to have a finer-grained comparison with the previous state. This is not currently surfaced in the output of cargo and still requires `RUST_LOG=cargo::ops::cargo_rustc::fingerprint=info` but if it turns out to be useful we can perhaps surface the output. There are performance considerations here to ensure that a noop build is still quite speedy for a few reasons: 1. JSON decoding is slow (these are just big structures to decode) 2. Each fingerprint stores all recursive fingerprints, so we can't just "vanilla decode" as it would decode O(n^2) items 3. Hashing is actually somewhat nontrivial for this many items here and there, so we still need as much memoization as possible. To ensure that builds are just as speedy tomorrow as they are today, a few strategies are taken: * The same fingerprint strategy is used today as a "first line of defense" where a small text file with a string contains the "total fingerprint" hash. A separately stored file then contains the more detailed JSON structure of the old fingerprint, and that's only decoded if there's a mismatch of the short hashes. The rationale here is that most crates don't need to be rebuilt so we shouldn't decode JSON, but if it does need to be rebuilt then the work of compiling far dwarfs the work of decoding the JSON. * When encoding a full fingerprint as JSON we don't actually include any dependencies, just the resolved u64 of them. This helps the O(n^2) problem in terms of decoding time and storage space on the filesystem. * Short hashes continue to be memoized to ensure we don't recompute a hash if we've already done so (e.g. shared dependencies). Overall, when profiling with Servo, this commit does not regress noop build times, but should help diagnose why crates are being rebuilt hopefully! Closes #2011
- Loading branch information