Internal compiler error: building dynamic library is unsupported on Emscripten

[kubkon]

Apologies if this is a duplicate, but I couldn't find a similar issue. Anyhow, the issue appears to be caused when trying to build for wasm32-unknown-emscripten target and including a lib crate with crate-type = ["rlib", "staticlib", "cdylib"] causes the following compiler panic:

thread 'rustc' panicked at 'src/librustc_codegen_ssa/back/linker.rs:878: building dynamic library is unsupported on Emscripten', src/librustc/util/bug.rs:37:26
stack backtrace:
   0:        0x1082d6c55 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h9462d4b5888539ab
   1:        0x10830a510 - core::fmt::write::hfc4799bb5f5713e1
   2:        0x1082c977b - std::io::Write::write_fmt::h688ebb99d695121f
   3:        0x1082db053 - std::panicking::default_hook::{{closure}}::h46807e40cc5c9ec9
   4:        0x1082dad5a - std::panicking::default_hook::hfa3fa9913244654c
   5:        0x1025301e2 - rustc_driver::report_ice::h37932e7c4f1fba7f
   6:        0x1082db84c - std::panicking::rust_panic_with_hook::hb7e09606b75e30c0
   7:        0x105974ec5 - std::panicking::begin_panic::hcf494bfc97f475ef
   8:        0x105a61c3c - rustc::util::bug::opt_span_bug_fmt::{{closure}}::h826c2d7833f6f58e
   9:        0x105a503ba - rustc::ty::context::tls::with_opt::{{closure}}::h50671f3cc14a88db
  10:        0x105a5036c - rustc::ty::context::tls::with_opt::ha2373df8753b454e
  11:        0x105a61b68 - rustc::util::bug::opt_span_bug_fmt::h894addd94d24f6bd
  12:        0x105a61abb - rustc::util::bug::bug_fmt::he4d8ec493683828c
  13:        0x10501fa7d - <rustc_codegen_ssa::back::linker::EmLinker as rustc_codegen_ssa::back::linker::Linker>::build_dylib::hd75fbf4112e25780
  14:        0x102753c1e - rustc_codegen_ssa::back::link::link_natively::h710fa9b0482dda87
  15:        0x102751ed2 - rustc_codegen_ssa::back::link::link_binary::h40e7f85745f8fa06
  16:        0x1028a20d7 - <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link::{{closure}}::h48ab3e569fa83cb0
  17:        0x102898d37 - rustc::util::common::time::h0fa921a50246e4dc
  18:        0x10284d7ce - <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link::hff6da34c9bbbb753
  19:        0x102679dcd - rustc_interface::queries::Linker::link::h892d2dbb5d6c10b7
  20:        0x10250011c - rustc_interface::interface::run_compiler_in_existing_thread_pool::ha1b7eef4d8bad1d7
  21:        0x1024d1479 - scoped_tls::ScopedKey<T>::set::he6a052ea9f345ec5
  22:        0x10254c415 - syntax::with_globals::h445b4296ae442554
  23:        0x1024d83cd - std::sys_common::backtrace::__rust_begin_short_backtrace::he6da9f1213f05905
  24:        0x1082eb00f - __rust_maybe_catch_panic
  25:        0x1024e4bc7 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h62a536cce0774879
  26:        0x1082bb66e - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h88795daf010cbf80
  27:        0x1082e9f9e - std::sys::unix::thread::Thread::new::thread_start::h731ad860c1aa479e
  28:     0x7fff63737e65 - _pthread_start

error: internal compiler error: unexpected panic

After further investigation, removing the cdylib from crate-type removes the error.

The weird thing is that this issue only seems to appear when building the main crate with cargo rustc instead of cargo build (the reason here involved passing in additional compiler flags). Also, the issue seems to concern all toolchains: 1.40.0-stable, 1.41.0-beta and 1.42.0-nightly.

Anyhow, the error can be replicated by running a build.sh script in wasmtime/crates/wasi-common/js-polyfill crate. For reference, the PR introducing this bit of functionality into wasmtime is bytecodealliance/wasmtime#720.

[jonas-schievink]

Seems like this is just a panic that should instead be a fatal error

[kubkon]

@jonas-schievink hmm, but does that mean if I'd like to use a lib which specifies itself as a dynamic lib using crate-type=[cdylib] amongst others in a binary cross-compiled to Emscripten, I will be greeted with a fatal error unless I remove the cdylib crate-type annotation?

[jonas-schievink]

No, only the cases causing the ICE will instead report a normal error. But it looks like the panic was never supposed to be reached in the first place, so there might be an additional bug.

[kubkon]

Ah, OK, makes sense. Thanks for the clarification!

[pnkfelix]

triage: P-medium, at least to get rid of the ICE and have it signal a fatal error instead. (But the note about there being a potential additional bug does make me wonder what else may be happening here.)

[kubkon]

@pnkfelix @jonas-schievink If it's not a nuisance to you guys, with some mentoring/guidance, I'd be more than happy than to dig in into the compiler and narrow down the cause and, hopefully, help out in providing a fix for this.

[codewiz]

This is happening to me when building the instant crate for wasm32-unknown-emscripten.
Removing cdylib as suggested by @kubkon does indeed fix the problem.

What should crate authors do for portability to emscripten?

[tanis2000]

This is happening to me as well in a project that creates both a binary and libraries. The definition in cargo.toml is like this:

[lib]
name = "minigame"
path = "src/lib.rs"
crate-type = ["staticlib", "cdylib", "rlib"]

[[bin]]
name = "main"
path = "src/main.rs"

Is there any way to tell cargo build to ignore the lib part and just build the bin?

[ehuss]

This seems to have been fixed (or at least changed) via #71804 so that it no longer errors.

[Enselic]

Triage: No error at all, or just no ICE? Would be great if someone could provide update step-by-step instructions on how to reproduce. When I try I get unrelated build errors:

git clone https://github.com/kubkon/wasmtime.git wasmtime-kubkon
cd wasmtime-kubkon
git checkout origin/js-poly
cd crates/wasi-common/js-polyfill
./build.sh

    Updating crates.io index
error: failed to get `witx` as a dependency of package `wig v0.7.0 (/home/martin/src/wasmtime-kubkon/crates/wasi-common/wig)`
    ... which satisfies path dependency `wig` (locked to 0.7.0) of package `wasi-common v0.7.0 (/home/martin/src/wasmtime-kubkon/crates/wasi-common)`
    ... which satisfies path dependency `wasi-common` (locked to 0.7.0) of package `polyfill v0.1.0 (/home/martin/src/wasmtime-kubkon/crates/wasi-common/js-polyfill)`

Caused by:
  failed to load source for dependency `witx`

Caused by:
  Unable to update /home/martin/src/wasmtime-kubkon/crates/wasi-common/WASI/tools/witx

Caused by:
  failed to read `/home/martin/src/wasmtime-kubkon/crates/wasi-common/WASI/tools/witx/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

[bjorn3]

You probably need to do git submodule update --init.

[kubkon]

Triage: No error at all, or just no ICE? Would be great if someone could provide update step-by-step instructions on how to reproduce. When I try I get unrelated build errors:

git clone https://github.com/kubkon/wasmtime.git wasmtime-kubkon
cd wasmtime-kubkon
git checkout origin/js-poly
cd crates/wasi-common/js-polyfill
./build.sh

   Updating crates.io index
error: failed to get `witx` as a dependency of package `wig v0.7.0 (/home/martin/src/wasmtime-kubkon/crates/wasi-common/wig)`
   ... which satisfies path dependency `wig` (locked to 0.7.0) of package `wasi-common v0.7.0 (/home/martin/src/wasmtime-kubkon/crates/wasi-common)`
   ... which satisfies path dependency `wasi-common` (locked to 0.7.0) of package `polyfill v0.1.0 (/home/martin/src/wasmtime-kubkon/crates/wasi-common/js-polyfill)`

Caused by:
 failed to load source for dependency `witx`

Caused by:
 Unable to update /home/martin/src/wasmtime-kubkon/crates/wasi-common/WASI/tools/witx

Caused by:
 failed to read `/home/martin/src/wasmtime-kubkon/crates/wasi-common/WASI/tools/witx/Cargo.toml`

Caused by:
 No such file or directory (os error 2)

Hi, I won't be of much help sadly as I have effectively moved on from Wasm and directed my focus elsewhere. I am happy to leave the issue open however given that more folks have also experienced this issue.

[Enselic]

Hi, I won't be of much help sadly

Hello, no worries! I totally understand. I agree that the ideal way forward is to keep this issue open until the underlying problem is fixed (or confirmed fixed already) and a regression test has been added (or confirmed added already).

However, without a way to reproduce the original ICE, there is no feasible way to bisect/investigate. Thanks to the hint from bjorn3 I got a bit longer in running ./build.sh:

docker run -it ubuntu:23.04

apt-get update -y && apt-get install -y build-essential curl git libssl-dev pkg-config
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source "$HOME/.cargo/env"   
rustup install nightly
rustup target add wasm32-unknown-emscripten --toolchain nightly
git clone https://github.com/kubkon/wasmtime.git wasmtime-kubkon
cd wasmtime-kubkon
git submodule update --init
git checkout origin/js-poly
cd crates/wasi-common/js-polyfill
./build.sh

What adjustments do I need to make to the above commands to reproduce the original ICE? Use an older nightly I suppose, but which one, and configured how? ./build.sh also needs to be changed because now it is hardcoded to use +nightly.

When I run the above commands I get a bunch of errors but no ICE. The first errors are:

error[E0412]: cannot find type `StructDatatype` in crate `witx`
   --> /wasmtime-kubkon/crates/wasi-common/wig/src/raw_types.rs:220:31
    |
220 | fn struct_has_union(s: &witx::StructDatatype) -> bool {
    |                               ^^^^^^^^^^^^^^ not found in `witx`

[Enselic]

Triage: Closing as not reproducible. Feel free to reopen if a way for others to reproduce is found!