compiler: capture rustc error output if extra-verbose

[infinity0]

The Debian rustc 1.33.0 build for mips* is failing with an error message like:

error: Could not compile `rustc`.

Caused by:
  compiler stdout is not empty: `Did not run successfully: Ok(ExitStatus(ExitStatus(6)))`

Caused by:
  failed to parse process output: `/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/bootstrap/debug/rustc --crate-name rustc src/librustc/lib.rs --color never --error-format json --crate-type dylib --emit=dep-info,link -C prefer-dynamic -C opt-level=2 -C metadata=bfb81208efc27141 -C extra-filename=-bfb81208efc27141 --out-dir /<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps --target mips-unknown-linux-gnu -L dependency=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps -L dependency=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/release/deps --extern arena=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libarena-e8c65afbe726ff70.so --extern backtrace=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libbacktrace-2f5f65b9ddc75a05.rlib --extern bitflags=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libbitflags-8dd4ee202268a284.rlib --extern byteorder=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libbyteorder-fbd1c0ae90e2cd44.rlib --extern chalk_engine=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libchalk_engine-35602a768e6452e8.rlib --extern flate2=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libflate2-120bd7acde379fe3.rlib --extern fmt_macros=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libfmt_macros-fcab36e5bacbd407.so --extern graphviz=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libgraphviz-ee68f0581b62bb8f.so --extern jobserver=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libjobserver-e35dd054e6112952.rlib --extern lazy_static=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/liblazy_static-85c1fe6037845491.rlib --extern log=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/liblog-dfbc8189aebd7366.rlib --extern parking_lot=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libparking_lot-90c4822847430ee9.rlib --extern polonius_engine=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libpolonius_engine-4e09b50c9ae29d08.rlib --extern rustc_rayon=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/librustc_rayon-0deb924f505c90c6.rlib --extern rustc_rayon_core=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/librustc_rayon_core-b99550ddc9076c45.rlib --extern rustc_apfloat=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/librustc_apfloat-6c7f2e1b035568de.rlib --extern rustc_data_structures=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/librustc_data_structures-31ec700018f56e4e.so --extern rustc_errors=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/librustc_errors-f99ed2d17492a67b.so --extern rustc_fs_util=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/librustc_fs_util-377332f92f52575f.so --extern rustc_target=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/librustc_target-9d4db7ca3bc869c2.so --extern scoped_tls=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libscoped_tls-6436cfae160256aa.rlib --extern serialize=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libserialize-c6ad2a7caead98fa.so --extern serialize=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libserialize-c6ad2a7caead98fa.rlib --extern smallvec=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libsmallvec-a27a121e841795c7.rlib --extern syntax=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libsyntax-c32de7485bbd4143.so --extern syntax_pos=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libsyntax_pos-88e1c54e0b12c9ec.so --extern tempfile=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/deps/libtempfile-26cae9afe9266eb2.rlib -C link-args=-Wl,-z,relro --remap-path-prefix=/<<BUILDDIR>>/rustc-1.33.0+dfsg1=/usr/src/rustc-1.33.0 -L native=/<<BUILDDIR>>/rustc-1.33.0+dfsg1/build/mips-unknown-linux-gnu/stage0-rustc/mips-unknown-linux-gnu/release/build/backtrace-sys-579f867ff9d4d857/out -L native=/usr/lib/mips-linux-gnu` (exit code: 0)
command did not execute successfully: "/usr/bin/cargo" "build" "--target" "mips-unknown-linux-gnu" "-j" "4" "-v" "-v" "--release" "--features" "" "--manifest-path" "/<<BUILDDIR>>/rustc-1.33.0+dfsg1/src/rustc/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101

It does not tell me what the actual error output of rustc is, making it very difficult to debug the problem.

This PR makes cargo capture rustc output and include it in the error message, if extra_verbose is on.

[rust-highfive]

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

[ehuss]

I'm not sure I understand this change. All of stderr and stdout is forwarded to the output (in latest nightly). This will just repeat what was already printed. Also, things have recently changed so that stdout is forwarded (it does not cause a failure). Is there some reason you need the output repeated?

[infinity0]

Looks like I was confused by the error message "Did not run successfully: Ok(ExitStatus(ExitStatus(6)))" which actually comes from rustc's bootstrap rustc-wrapper. It's unclear if that wrapper is dropping stdout/stderr or if it's actually empty, however.

[ehuss]

That is curious. AFAIK, the bootstrap wrapper disables JSON output and forwards stdout/stderr to cargo. In the case of the version you're using, stderr would just pass through, but it would fail on the first line on stdout (the latest cargo won't do that). So I would be surprised if the stderr messages are lost.

Error code 6 also seems suspicious. I've never seen rustc exit with such a code. Is that maybe SIGABRT?

[infinity0]

So, stderr is definitely getting lost. I ran the command manually and it gives me "LLVM ERROR: out of memory" on stderr with the shell adding "Aborted", which seems to be the same problem as in our automated build log (and the same problem as rust-lang/rust#56888). But this seems more like a bug in rustc's bootstrap rather than cargo so I'll investigate some more and probably close this PR soon.

[alexcrichton]

The OP looks like this line which indeed should be fixed on Cargo's master branch, and I think I agree with @ehuss that this in theory shouldn't be needed today, although it would be best to confirm within the context of this issue

[bors]

☔ The latest upstream changes (presumably #6933) made this pull request unmergeable. Please resolve the merge conflicts.

[infinity0]

Looks like the issue is fixed in rustc 1.34.2, we do get "LLVM out of memory" in the build logs again, so closing.