Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linking a static native library to a dynamic rust library doesn't necessarily actually link #15460

Closed
alexcrichton opened this issue Jul 5, 2014 · 5 comments · Fixed by #16110
Labels
A-linkage Area: linking into static, shared libraries and binaries

Comments

@alexcrichton
Copy link
Member

// foo.c
void foo() {}
// foo.rs
#![crate_type = "dylib"]
#[link(name = "foo", kind = "static")]
extern {
    pub fn foo();
}
// bar.rs
extern crate foo;
fn main() { unsafe { foo::foo() } }
$ gcc -c foo.c                         
$ ar crus libfoo.a foo.o    
$ rustc foo.rs -L.                  
$ rustc bar.rs -L.      
error: linking with `cc` failed: exit code: 1
note: cc '-m64' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-o' 'bar' 'bar.o' '-Wl,-force_load,/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib/libmorestack.a' '-nodefaultlibs' '-Wl,-dead_strip' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-lnative-1fb5e2c0-0.11.0-pre' '-L' '/Users/acrichton/tmp' '-lfoo-aab0fa9e-0.0' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-lstd-59beb4f7-0.11.0-pre' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-lsync-305341d2-0.11.0-pre' '-L' '/Users/acrichton/code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-lrustrt-d8560cb2-0.11.0-pre' '-L' '.' '-L' '/Users/acrichton/tmp/.rust' '-L' '/Users/acrichton/tmp' '-L' '/Users/acrichton/.rust' '-lSystem' '-lpthread' '-lc' '-lm' '-Wl,-rpath,@loader_path/../code/rust/lib/rustlib/x86_64-apple-darwin/lib' '-Wl,-rpath,@loader_path/.' '-Wl,-rpath,/usr/local/lib/rustlib/x86_64-apple-darwin/lib' '-lcompiler-rt'
note: ld: warning: directory not found for option '-L/Users/acrichton/tmp/.rust'
Undefined symbols for architecture x86_64:
  "_foo", referenced from:
      main::hdcb342faf114af9efaa::v0.0 in bar.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: aborting due to previous error

The problem is that if the dynamic library doesn't actually reference any symbols from the native library then the native library is stripped from the command line.

Possible solutions

  • Stop using -Wl,--as-needed
  • Pass -Wl,--whole-archive for immediately dependent static libraries
  • For all extern blocks with a #[link(kind = "static")], generate a reference to all symbols in the extern block to force the library to be linked. (in some sort of 0-cost manner)
@alexcrichton alexcrichton changed the title Linking a static native library to a dynamic rust library doesn't necessary actually link Linking a static native library to a dynamic rust library doesn't necessarily actually link Jul 5, 2014
@o11c
Copy link

o11c commented Jul 8, 2014

Should retitle to indicate that it's not just about static native libraries.

@alexcrichton
Copy link
Member Author

In your test case you are using #[link_args] which is a feature-gated and unsupported way to link to dynamic libraries.

This is not a problem using dynamic libraries through the #[link] attribute.

@o11c
Copy link

o11c commented Jul 8, 2014

Given that #[link] is fundamentally wrong in nearly all cases, #[link_args] will have to be blessed sooner or later.

alexcrichton added a commit to alexcrichton/rust that referenced this issue Aug 4, 2014
As discovered in rust-lang#15460, a particular #[link(kind = "static", ...)] line is not
actually guaranteed to link the library at all. The reason for this is that if
the external library doesn't have any referenced symbols in the object generated
by rustc, the entire library is dropped by the linker.

For dynamic native libraries, this is solved by passing -lfoo for all downstream
compilations unconditionally. For static libraries in rlibs this is solved
because the entire archive is bundled in the rlib. The only situation in which
this was a problem was when a static native library was linked to a rust dynamic
library.

This commit brings the behavior of dylibs in line with rlibs by passing the
--whole-archive flag to the linker when linking native libraries. On OSX, this
uses the -force_load flag. This flag ensures that the entire archive is
considered candidate for being linked into the final dynamic library.

This is a breaking change because if any static library is included twice in the
same compilation unit then the linker will start emitting errors about duplicate
definitions now. The fix for this would involve only statically linking to a
library once.

Closes rust-lang#15460
[breaking-change]
bors added a commit that referenced this issue Aug 4, 2014
As discovered in #15460, a particular #[link(kind = "static", ...)] line is not
actually guaranteed to link the library at all. The reason for this is that if
the external library doesn't have any referenced symbols in the object generated
by rustc, the entire library is dropped by the linker.

For dynamic native libraries, this is solved by passing -lfoo for all downstream
compilations unconditionally. For static libraries in rlibs this is solved
because the entire archive is bundled in the rlib. The only situation in which
this was a problem was when a static native library was linked to a rust dynamic
library.

This commit brings the behavior of dylibs in line with rlibs by passing the
--whole-archive flag to the linker when linking native libraries. On OSX, this
uses the -force_load flag. This flag ensures that the entire archive is
considered candidate for being linked into the final dynamic library.

This is a breaking change because if any static library is included twice in the
same compilation unit then the linker will start emitting errors about duplicate
definitions now. The fix for this would involve only statically linking to a
library once.

Closes #15460
[breaking-change]
@Firstyear
Copy link
Contributor

Hi there, sorry to re-open this many years after, but I can reproduce this on 1.47 stable and 1.48 nightly with the repository found here: https://github.com/Firstyear/demo_r_link

Using cargo build then inspecting the resulting .so, "wrapper_add" is not present. Using the linking trick from @steveklabnik's closed issue can force symbols to be included from the static archive. This is using cc and cbindgen, so this "should work". Any advice would be most appreciated.

@steveklabnik
Copy link
Member

@Firstyear in general, especially with a bug this old, it's better to open a new issue, and mention this one in it. A lot has changed since 2014 and most of the details here won't be relevant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linking into static, shared libraries and binaries
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants