Failure to load LLVM-compiled WASM: undefined symbol: wasmer_raise_trap

[dotxlem]

Describe the bug

I have written an executable which compiles WASM via LLVM and serializes the module to disk.

I am attempting to cross-compile to Amazon Linux via target triple x86_64-linux-unknown, from macOS 11.1 (uname: Darwin xlembox.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec 2 20:39:59 PST 2020; root:xnu-7195.60.75~1/RELEASE_X86_64 x86_64).

On the Linux side, another executable attempts to load the binary to run via the Native engine in headless mode:

let store = Store::new(&Native::headless().engine());
let module = unsafe { Module::deserialize_from_file(&store, file_path) }.unwrap();

Deserialize panics on the unwrap with

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: CorruptedBinary("Library loading failed: /var/task/put-iomod-at.wasm.bin: undefined symbol: wasmer_raise_trap")', backends/aws-lambda/host/src/wasm.rs:28:78

Steps to reproduce

1. Cross-compile WASM to module binary via LLVM and serialize the result.
2. Load & deserialize the binary on the compilation target

Expected behavior

The module loads and executes via the native headless engine.

Actual behavior

Deserialize panics with CorruptedBinary error.

Additional context

This process works when using Cranelift instead of LLVM.

Snippet of code I use to cross-compile:

let mut compiler = LLVM::default();
let triple = Triple::from_str("x86_64-linux-unknown").unwrap();
let mut cpuid = CpuFeature::set();
cpuid.insert(CpuFeature::from_str("sse2").unwrap());
cpuid.insert(CpuFeature::from_str("avx2").unwrap());
let store = Store::new(&Native::new(compiler)
    target(Target::new(triple, cpuid))
    .engine()
);

let wasm_bytes = match fs::read(wasm_path) {
    Ok(bytes) => bytes,
    Err(err) => panic!(err.to_string()),
};
let module = Module::new(&store, wasm_bytes).unwrap();
let module_bytes = module.serialize().unwrap();

[nlewycky]

The final binary that uses engine-native must link with a flag that exposes the wasmer VM functions to the shared object that we load. Here's our .cargo/cargo.toml: https://github.com/wasmerio/wasmer/blob/master/.cargo/config.toml . Try adding that to your project!

[dotxlem]

Thanks @nlewycky! Unfortunately I haven't been able to test this yet -- I'm running into issues getting LLVM running in my build container.

Will report back when I'm up and running again :)

[nlewycky]

Is that .cargo/config for the program that is being compiled into a .wasm? You need to update the native program that is using wasmer as a library, not the build of the .wasm code.

This .cargo/config needs to go on the Linux-side program that uses wasmer as a library to load the compiled .so file that was produced by compiling the .wasm:

[target.'cfg(target_os = "linux")']
rustflags = [
    # Put the VM functions in the dynamic symbol table.
    "-C", "link-arg=-Wl,-E",
]

[dotxlem]

Sorry I deleted my comment, I realized after that it likely wasn't supposed to go on the .wasm

However I also have it on the host binary which loads the .wasm, with the same result :(

[nlewycky]

It sounds like you have two binaries, one on a mac which loads the .wasm and compiles to .so, and another on linux which loads the .so. Is that right? If so, this needs to go on the Linux one. If that still isn't working, I'll check for a bug where we pick the symbols names, maybe we're accidentally picking host names instead of target names or something like that.

[dotxlem]

I can confirm that I have the config.toml defined for the linux binary; is there a way to inspect the binary itself to make sure it's worked?

If it's helpful at all, the linux binary is running in Docker under Alpine 3.13 & linked against glibc-2.32

[nlewycky]

I could use some help reproducing the issue, I've set up Alpine 3.14 with glibc 2.33 using packages from https://github.com/sgerrand/alpine-pkg-glibc but building with cargo build --target x86_64-unknown-linux-gnu produces a binary that uses ld-musl and has an unrelated error involving mapping too much memory.

[Hywan]

This issue may be fixed by #2475.

[syrusakbary]

We could solve this by having libcalls in the compilers be resolved by traversing the VMContext rather than using a static pointer at compile time

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Feel free to reopen the issue if it has been closed by mistake.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Feel free to reopen the issue if it has been closed by mistake.