Bootstrap sometimes fails to build miri because of "nightly" feature confusion

[RalfJung]

Step to reproduce:

rm build/host/stage0* -rf
./x.py check library/std compiler/rustc src/tools/miri

Now Miri fails to build with >1800 errors. The first ones are:

error[E0464]: multiple candidates for `rmeta` dependency `rustc_ast` found
  --> src/tools/miri/src/lib.rs:53:1
   |
53 | extern crate rustc_ast;
   | ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_ast-11854d11a3563889.rmeta
   = note: candidate #2: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_ast-feb80e66ce1c74d4.rmeta

error[E0464]: multiple candidates for `rmeta` dependency `rustc_const_eval` found
  --> src/tools/miri/src/lib.rs:54:1
   |
54 | extern crate rustc_const_eval;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_const_eval-078a7b0d61866866.rmeta
   = note: candidate #2: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_const_eval-2fa8de12acbc8979.rmeta

error[E0464]: multiple candidates for `rmeta` dependency `rustc_data_structures` found
  --> src/tools/miri/src/lib.rs:55:1
   |
55 | extern crate rustc_data_structures;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_data_structures-93605ddb6ab78ed8.rmeta
   = note: candidate #2: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_data_structures-e65a21da042ec502.rmeta

error[E0464]: multiple candidates for `rmeta` dependency `rustc_errors` found
  --> src/tools/miri/src/lib.rs:56:1
   |
56 | extern crate rustc_errors;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_errors-792f00940afc5752.rmeta
   = note: candidate #2: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_errors-ddd5d12697277ea9.rmeta

error[E0464]: multiple candidates for `rmeta` dependency `rustc_hir` found
  --> src/tools/miri/src/lib.rs:57:1
   |
57 | extern crate rustc_hir;
   | ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_hir-37bcaf20f0254cb3.rmeta
   = note: candidate #2: /home/r/src/rust/rustc.2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_hir-6f8ad5718a5b342e.rmeta

However, if I now run ./x.py check miri, everything is fine. rm build/host/stage0* -rf && ./x.py check miri is also fine. Something about explicitly giving the three paths to build is throwing bootstrap off. And even more strangely, the same sequence of commands worked fine before #116828.

Cc @rust-lang/bootstrap

[onur-ozkan]

compiler/rustc_index_macros should be compiled with its default feature nightly. However, in rustc_index, this feature disabled by default. When compiling these specific paths (library/std compiler/rustc src/tools/miri), rustc_index_macros is not built with its default feature.

A quick workaround for your case would be adding compiler/rustc_index_macros path in x check (like x check library/std compiler/rustc compiler/rustc_index_macros src/tools/miri).

[RalfJung]

FWIW I cannot reproduce the problem any more. Has this been fixed?

[RalfJung]

Actually never mind the issue still exists

[RalfJung]

@onur-ozkan

However, in rustc_index, this feature disabled by default.

I don't understand this explanation. Looking at the Cargo.toml, we have

rustc_index_macros = { path = "../rustc_index_macros", default-features = false }

and

[features]
# tidy-alphabetical-start
default = ["nightly"]
nightly = [
    "dep:rustc_serialize",
    "dep:rustc_macros",
    "rustc_index_macros/nightly",
]
rustc_randomized_layouts = []
# tidy-alphabetical-end

So by default, we do enable rustc_index_macros/nightly.

Why does adding compiler/rustc_index_macros even make any difference here? I can't find any code in bootstrap that would special-case this crate. This is all quite strange to me.

[RalfJung]

Cc @compiler-errors

The work-around for this works, but it makes the x invocation in RA so long that RA no longer shows progress during the build in vscode. So it'd be nice to get this fixed properly; something about this entire "nightly" feature setup is just not working correctly currently.

[compiler-errors]

what am i being ccd for

[RalfJung]

@compiler-errors you wrote some of the commits adding this "nightly" feature everywhere (e.g. 4506681). Somehow that is causing issues with building Miri: some rustc crates end up being built twice with different feature sets, and then the extern crate in Miri doesn't know which crate to load. I don't understand how this nightly feature is being enabled/disabled where, so I figured maybe you can help.

[RalfJung]

I think I found another work-around: ./x check library compiler miri also works. It's just compiler/rustc that causes problems. I have no idea why...

[compiler-errors]

not totally sure, but sounds like a bootstrap issue 🤔

[onur-ozkan]

It's more of a cargo thing (I am not sure if this is an expected behaviour or not). When default-features = false is set, cargo can't propagate "nightly" feature which is a default feature for compiler/rustc_index_macros/Cargo.toml but disabled with default-features = false in compiler/rustc_index/Cargo.toml.

See #134526 for the fix.

cc @rust-lang/cargo