Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extern Swift declaration not public #241

Closed
tmolitor-stud-tu opened this issue Sep 2, 2023 · 12 comments
Closed

extern Swift declaration not public #241

tmolitor-stud-tu opened this issue Sep 2, 2023 · 12 comments

Comments

@tmolitor-stud-tu
Copy link

We are building a library that has swift code that should be called by rust in an extra swift file (besides the auto generated ones).
But we keep getting linker errors that __swift_bridge__$rust_handler could not be found.
Until we make the auto-generated function public:

@_cdecl("__swift_bridge__$rust_handler")
public func __swift_bridge__rust_handler (_ text: UnsafeMutableRawPointer) {
    rust_handler(text: RustString(ptr: text))
}

Could you fix the code autogenerator to make these public by default (or at least introduce some compile-time switch to do that)?

@NiwakaDev
Copy link
Collaborator

Could you share the minimal example?

@tmolitor-stud-tu
Copy link
Author

See this zip file for a minimal example.
The build-rust.sh script is based upon https://github.com/chinedufn/swift-bridge/tree/master/book/src/building/swift-packages

The generated code looks like this:

import RustXcframework
public func test() {
    __swift_bridge__$test()
}
@_cdecl("__swift_bridge__$swift_handler")
func __swift_bridge__swift_handler (_ text: UnsafeMutableRawPointer) {
    swift_handler(text: RustString(ptr: text))
}

Thus the linker will report __swift_bridge__$swift_handler as missing once the generated LibTestBridge package is linked to from any swift project.
Adding public in front of the __swift_bridge__swift_handler definition will solve that issue.

That's because of swifts access control defaulting to internal which prevents the binary RustXcframework.xcframework from accessing the __swift_bridge__swift_handler supposed to be exposed as symbol __swift_bridge__$swift_handler in the object file generated by the swift compiler, but the swift compiler does not expose this symbol.

The swift compiler does not expose the symbol because the internal access control means this function should only be accessible by the swift module it resides in, not by any other modules (linked in).

@tmolitor-stud-tu
Copy link
Author

If you point me in the right direction, I can create a pull request fixing this :)

@NiwakaDev
Copy link
Collaborator

NiwakaDev commented Sep 22, 2023

@tmolitor-stud-tu
Sorry for the delayed response and thank you for sharing the example.

If you point me in the right direction, I can create a pull request fixing this :)

I'll dive into this issue next weekend because I'm a bit busy right now.

@NiwakaDev
Copy link
Collaborator

NiwakaDev commented Sep 29, 2023

@tmolitor-stud-tu

public func test() {
    __swift_bridge__$test()
}
@_cdecl("__swift_bridge__$swift_handler")
func __swift_bridge__swift_handler (_ text: UnsafeMutableRawPointer) {
    swift_handler(text: RustString(ptr: text))
}

In your example, it seems like __swift_bridge__swift_handler isn't called in other swift modules. Do you want to call this method in other swift modules?

I think we need to share an example which can reproduce this error.

@NiwakaDev
Copy link
Collaborator

NiwakaDev commented Sep 29, 2023

As far as I remember, this method is used on the Rust side. So, other swift modules shouldn't call this.

@tmolitor-stud-tu
Copy link
Author

As far as I remember, this method is used on the Rust side. So, other swift modules shouldn't call this.

Exactly, but the binary rust XCFramework has an unresolved symbol (__swift_bridge__$swift_handler) because the __swift_bridge__swift_handler swift function generated by swift-bridge is not public and thus its symbol won't be exported.
This means the linker sees two binaries, one containing the rust stuff that wants a symbol named __swift_bridge__$swift_handler and the other (the swift one) not exporting that symbol even though it should (that's the whole point of the __swift_bridge__swift_handler function in the first place: being there to be called by rust code).

Please note: even though I did not include swift code containing the swift_handler function, that linker problem is unrelated to this. Just create a swift file inside the test-bridge/generated directory containing that function.

Sidenote: a full (non-minimal) example can be found over here: https://github.com/monal-im/Monal/tree/develop/rust
Please note the sed invocation at the end of the build script fixing this issue: https://github.com/monal-im/Monal/blob/develop/rust/build-rust.sh#L60

@NiwakaDev
Copy link
Collaborator

NiwakaDev commented Sep 30, 2023

@tmolitor-stud-tu

What linker do you use?

a full (non-minimal) example can be found over here: https://github.com/monal-im/Monal/tree/develop/rust
Please note the sed invocation at the end of the build script fixing this issue: https://github.com/monal-im/Monal/blob/develop/rust/build-rust.sh#L60

By the way, I added the your package to my minimal project like this:
image1
image2
image3

Even though I don't put public in front of __swift_bridge__rust_panic_handler method, Xcode's default linker could link them. Am I missing something?

Environment

Xcode: Version 14.3.1

@NiwakaDev
Copy link
Collaborator

NiwakaDev commented Sep 30, 2023

This means the linker sees two binaries, one containing the rust stuff that wants a symbol named swift_bridge$swift_handler and the other (the swift one) not exporting that symbol even though it should (that's the whole point of the __swift_bridge__swift_handler function in the first place: being there to be called by rust code).

As you can see the immediately above comment, it seems like the program run, which means the linker could link all of the rust object files and swift object files.

@tmolitor-stud-tu
Copy link
Author

Well, I'm using Xcode Version 14.3.1, too.

The build command is this:
2023-09-30T17:46:08.3624550Z /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -c /Users/ci/Desktop/actions-runner/_work/Monal/Monal/rust/LibMonalRustSwiftBridge/Sources/LibMonalRustSwiftBridge/SwiftBridgeCore.swift /Users/ci/Desktop/actions-runner/_work/Monal/Monal/rust/LibMonalRustSwiftBridge/Sources/LibMonalRustSwiftBridge/monal-rust-swift-bridge.swift /Users/ci/Desktop/actions-runner/_work/Monal/Monal/rust/LibMonalRustSwiftBridge/Sources/LibMonalRustSwiftBridge/panichandling.swift -supplementary-output-file-map /Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/Objects-normal/x86_64/supplementaryOutputs-11 -target x86_64-apple-ios13.1-macabi -enable-objc-interop -stack-check -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -I /Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/BuildProductsPath/Alpha-maccatalyst -I /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -F /Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/BuildProductsPath/Alpha-maccatalyst -Fsystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/System/iOSSupport/System/Library/Frameworks -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -no-color-diagnostics -g -module-cache-path /Users/ci/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -swift-version 5 -enforce-exclusivity\=checked -O -D SWIFT_PACKAGE -D Xcode -serialize-debugging-options -empty-abi-descriptor -Xcc -working-directory -Xcc /Users/ci/Desktop/actions-runner/_work/Monal/Monal/rust/LibMonalRustSwiftBridge -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -Xcc -ivfsstatcache -Xcc /Users/ci/Library/Developer/Xcode/DerivedData/SDKStatCaches.noindex/macosx13.3-22E245-.sdkstatcache -Xcc -I/Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/swift-overrides.hmap -Xcc -I/Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/BuildProductsPath/Alpha-maccatalyst/include -Xcc -isystem -Xcc /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/System/iOSSupport/usr/include -Xcc -I/Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/DerivedSources-normal/x86_64 -Xcc -I/Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/DerivedSources/x86_64 -Xcc -I/Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/DerivedSources -Xcc -DSWIFT_PACKAGE -module-name LibMonalRustSwiftBridge -frontend-parseable-output -disable-clang-spi -target-sdk-version 16.4 -target-sdk-name macosx13.3 -prebuilt-module-cache-path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/prebuilt-modules/13.3 -enable-default-cmo -num-threads 8 -o /Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/Objects-normal/x86_64/SwiftBridgeCore.o -o /Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/Objects-normal/x86_64/monal-rust-swift-bridge.o -o /Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/IntermediateBuildFilesPath/LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/Objects-normal/x86_64/panichandling.o -index-unit-output-path /LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/Objects-normal/x86_64/SwiftBridgeCore.o -index-unit-output-path /LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/Objects-normal/x86_64/monal-rust-swift-bridge.o -index-unit-output-path /LibMonalRustSwiftBridge.build/Alpha-maccatalyst/LibMonalRustSwiftBridge.build/Objects-normal/x86_64/panichandling.o

And later on:
2023-09-30T17:45:49.9506860Z builtin-process-xcframework --xcframework /Users/ci/Desktop/actions-runner/_work/Monal/Monal/rust/LibMonalRustSwiftBridge/RustXcframework.xcframework --platform ios --environment macabi --target-path /Users/ci/Library/Developer/Xcode/DerivedData/Monal-gkigrzmwpngdmfcmvdmwolhwpmgz/Build/Intermediates.noindex/ArchiveIntermediates/Monal/BuildProductsPath/Alpha-maccatalyst

What commands did you use to build the rust binary? Just to double check: our build script uses sed to put public in front of the autogenerated swift method. If you used that, you obviously won't be able to reproduce this bug.

That said I'm almost certain that the swift compiler won't export any symbols of functions that are not public (because swift functions without explicit access control default to "internal").

In your test the binary rust xcframework seems to be seen as internal by the swift compiler, while in my case the compiler seems to see the rust framework as external.
I'm not sure, what build setting triggers this, but my guess would be dylib versus static lib.
We use dylibs at some points and dylibs seem to be stricter in what symbols they export: swiftlang/swift#43633

Regardless of the root cause of this: I really think that using public as access control specifier for functions that should be called by rust is the more robust way and should be preferred.
I don't know your codebase well enough to change them to public and create a PR, though.

@chinedufn
Copy link
Owner

#262 should fix this

@tmolitor-stud-tu
Copy link
Author

Thanks @jmp-0x7C0 !!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants