From 97f345adbec0144abb78683c54720c8e0a6b5cad Mon Sep 17 00:00:00 2001 From: Peter Collins Date: Tue, 3 Sep 2019 21:33:36 -0400 Subject: [PATCH] [XA.Build.Tasks] Add AndroidBinUtilsDirectory prop (#3567) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes: https://github.com/xamarin/xamarin-android/issues/3564 Install Visual Studio for Mac on macOS 10.15 Catalina Beta 7, and build a project. Expected results: it builds! Actual results: not so much: Task Parameter:AndroidSdkBuildToolsPath=…/android-sdk-macosx/build-tools/28.0.3/ (TaskId:273) [Native Linker] …/android-sdk-macosx/build-tools/28.0.3/aarch64-linux-android-ld --unresolved-symbols=ignore-in-shared-libs --export-dynamic -soname libxamarin-app.so -z relro -z noexecstack --enable-new-dtags --eh-frame-hdr -shared --build-id --warn-shared-textrel --fatal-warnings -o obj/Release/app_shared_libraries/arm64-v8a/libxamarin-app.so --fix-cortex-a53-843419 -m aarch64linux obj/Release/android/typemap.jm.arm64-v8a.o obj/Release/android/typemap.mj.arm64-v8a.o obj/Release/android/environment.arm64-v8a.o (TaskId:273) [Native Linker] …/android-sdk-macosx/build-tools/28.0.3/arm-linux-androideabi-ld --unresolved-symbols=ignore-in-shared-libs --export-dynamic -soname libxamarin-app.so -z relro -z noexecstack --enable-new-dtags --eh-frame-hdr -shared --build-id --warn-shared-textrel --fatal-warnings -o obj/Release/app_shared_libraries/armeabi-v7a/libxamarin-app.so -X -m armelf_linux_eabi obj/Release/android/typemap.jm.armeabi-v7a.o obj/Release/android/typemap.mj.armeabi-v7a.o obj/Release/android/environment.armeabi-v7a.o (TaskId:273) /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2869,3): error XA3001: Could not link native shared library: libxamarin-app.so Done executing task "LinkApplicationSharedLibraries" -- FAILED. The cause of the error is: 1. `libxamarin-app.so` is required (see decfbccf), which 2. Requires that we be able to execute `arm-linux-androideabi-ld`, and 3. macOS 10.15 Catalina no longer executes 32-bit binaries. As it happens™, the Android SDK Build-tools package still contains 32-bit native binaries, up through and including Build-tools 29.0.0. (The Android NDK *does* contain 64-bit binaries.) At this time, we don't know if there will *be* a Build-tools package which contains 64-bit binaries before Catalina drops. It is also moot: as of f0b1e8a5 we now redistribute *our own* `*-ld` and `*-strip` utilities, which are 64-bit. There is thus no need to use the Build-tools versions of these files. The problem here was that some of our tasks used the bundled NDK tools, while some of our tasks didn't, and instead required that they exist within `$(AndroidSdkBuildToolsPath)`, the Android SDK Build-tools directory. Add a new overridable `$(AndroidBinUtilsDirectory)` MSBuild property which contains the `*-as`, `*-ld`, and `*-strip` utilities needed for assembling and linking native code. The value of this new property defaults to inside the Xamarin.Android installation path, `$(MonoAndroidBinDirectory)\ndk\`. `$(AndroidBinUtilsDirectory)` is an overridable property because the NDK utilities are GPL'd, and if someone wishes to use their own version of these utilities, overriding `$(AndroidBinUtilsDirectory)` will be much easier than replacing files within the installation directory. The result is that builds now use reliably 64-bit binaries: Task "LinkApplicationSharedLibraries" [Native Linker] /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/Darwin/ndk/aarch64-linux-android-ld --unresolved-symbols=ignore-in-shared-libs --export-dynamic -soname libxamarin-app.so -z relro -z noexecstack --enable-new-dtags --eh-frame-hdr -shared --build-id --warn-shared-textrel --fatal-warnings -o obj/Release/app_shared_libraries/arm64-v8a/libxamarin-app.so --fix-cortex-a53-843419 -m aarch64linux obj/Release/android/typemap.jm.arm64-v8a.o obj/Release/android/typemap.mj.arm64-v8a.o obj/Release/android/environment.arm64-v8a.o [Native Linker] /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/Darwin/ndk/arm-linux-androideabi-ld --unresolved-symbols=ignore-in-shared-libs --export-dynamic -soname libxamarin-app.so -z relro -z noexecstack --enable-new-dtags --eh-frame-hdr -shared --build-id --warn-shared-textrel --fatal-warnings -o obj/Release/app_shared_libraries/armeabi-v7a/libxamarin-app.so -X -m armelf_linux_eabi obj/Release/android/typemap.jm.armeabi-v7a.o obj/Release/android/typemap.mj.armeabi-v7a.o obj/Release/android/environment.armeabi-v7a.o Done executing task "LinkApplicationSharedLibraries". --- Documentation/guides/BuildProcess.md | 10 ++++++++++ src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs | 5 +++-- .../Tasks/CompileNativeAssembly.cs | 12 +++++++----- .../Tasks/LinkApplicationSharedLibraries.cs | 6 +++--- .../Tasks/ResolveSdksTask.cs | 5 +++++ .../Xamarin.Android.Common.targets | 6 ++++-- 6 files changed, 32 insertions(+), 12 deletions(-) diff --git a/Documentation/guides/BuildProcess.md b/Documentation/guides/BuildProcess.md index 2731a5b3fdf..2fcaefaedd7 100644 --- a/Documentation/guides/BuildProcess.md +++ b/Documentation/guides/BuildProcess.md @@ -257,6 +257,16 @@ when packaging Release applications. Added in Xamarin.Android 6.1. +- **AndroidBinUtilsPath** – A path to a directory containing + the Android [binutils][binutils] such as `ld`, the native linker, + and `as`, the native assembler. These binaries a subset of the + Android NDK, and are bundled with the Xamarin.Android + installation. `$(MonoAndroidBinDirectory)\ndk\` by default. + + Added in Xamarin.Android 10.0. + +[binutils]: https://android.googlesource.com/toolchain/binutils/ + - **AndroidBuildApplicationPackage** – A boolean value that indicates whether to create and sign the package (.apk). Setting this value to `True` is equivalent to using the diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs b/src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs index 966dbaa5bcf..96023b75a05 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs @@ -70,7 +70,8 @@ public class Aot : AsyncTask public ITaskItem [] Profiles { get; set; } - public string ToolsDirectory { get; set; } + [Required] + public string AndroidBinUtilsDirectory { get; set; } [Output] public string[] NativeLibrariesReferences { get; set; } @@ -364,7 +365,7 @@ IEnumerable GetAotConfigs () int level = 0; string toolPrefix = EnableLLVM ? NdkUtil.GetNdkToolPrefix (AndroidNdkDirectory, arch, level = GetNdkApiLevel (AndroidNdkDirectory, AndroidApiLevel, arch)) - : Path.Combine (ToolsDirectory, "ndk", $"{NdkUtil.GetArchDirName (arch)}-"); + : Path.Combine (AndroidBinUtilsDirectory, $"{NdkUtil.GetArchDirName (arch)}-"); var toolchainPath = toolPrefix.Substring(0, toolPrefix.LastIndexOf(Path.DirectorySeparatorChar)); var ldFlags = string.Empty; if (EnableLLVM) { diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/CompileNativeAssembly.cs b/src/Xamarin.Android.Build.Tasks/Tasks/CompileNativeAssembly.cs index 13b369c97db..1b22a89cc34 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/CompileNativeAssembly.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/CompileNativeAssembly.cs @@ -30,6 +30,9 @@ sealed class Config [Required] public string WorkingDirectory { get; set; } + [Required] + public string AndroidBinUtilsDirectory { get; set; } + public override bool Execute () { try { @@ -134,7 +137,6 @@ bool RunAssembler (Config config) IEnumerable GetAssemblerConfigs () { - string sdkBinDirectory = Path.Combine (MonoAndroidHelper.GetOSBinPath (), "ndk"); foreach (ITaskItem item in Sources) { string abi = item.GetMetadata ("abi")?.ToLowerInvariant (); string prefix = String.Empty; @@ -142,24 +144,24 @@ IEnumerable GetAssemblerConfigs () switch (abi) { case "armeabi-v7a": - prefix = Path.Combine (sdkBinDirectory, "arm-linux-androideabi"); + prefix = Path.Combine (AndroidBinUtilsDirectory, "arm-linux-androideabi"); arch = AndroidTargetArch.Arm; break; case "arm64": case "arm64-v8a": case "aarch64": - prefix = Path.Combine (sdkBinDirectory, "aarch64-linux-android"); + prefix = Path.Combine (AndroidBinUtilsDirectory, "aarch64-linux-android"); arch = AndroidTargetArch.Arm64; break; case "x86": - prefix = Path.Combine (sdkBinDirectory, "i686-linux-android"); + prefix = Path.Combine (AndroidBinUtilsDirectory, "i686-linux-android"); arch = AndroidTargetArch.X86; break; case "x86_64": - prefix = Path.Combine (sdkBinDirectory, "x86_64-linux-android"); + prefix = Path.Combine (AndroidBinUtilsDirectory, "x86_64-linux-android"); arch = AndroidTargetArch.X86_64; break; diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/LinkApplicationSharedLibraries.cs b/src/Xamarin.Android.Build.Tasks/Tasks/LinkApplicationSharedLibraries.cs index ae043b7f4d8..54b54032f58 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/LinkApplicationSharedLibraries.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/LinkApplicationSharedLibraries.cs @@ -37,7 +37,7 @@ sealed class InputFiles public bool DebugBuild { get; set; } [Required] - public string AndroidSdkBuildToolsPath { get; set; } + public string AndroidBinUtilsDirectory { get; set; } public override bool Execute () { @@ -210,9 +210,9 @@ IEnumerable GetLinkerConfigs () linkerArgs.Add (QuoteFileName (file)); } - string ld = MonoAndroidHelper.GetExecutablePath (AndroidSdkBuildToolsPath, $"{NdkUtil.GetNdkToolchainPrefix (arch, false)}ld"); + string ld = MonoAndroidHelper.GetExecutablePath (AndroidBinUtilsDirectory, $"{NdkUtil.GetNdkToolchainPrefix (arch, false)}ld"); yield return new Config { - LinkerPath = Path.Combine (AndroidSdkBuildToolsPath, ld), + LinkerPath = Path.Combine (AndroidBinUtilsDirectory, ld), LinkerOptions = String.Join (" ", linkerArgs), OutputSharedLibrary = inputs.OutputSharedLibrary, }; diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/ResolveSdksTask.cs b/src/Xamarin.Android.Build.Tasks/Tasks/ResolveSdksTask.cs index 155827f04fc..d6d9bbbe713 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/ResolveSdksTask.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/ResolveSdksTask.cs @@ -59,6 +59,9 @@ public class ResolveSdks : Task [Output] public string MonoAndroidLibPath { get; set; } + [Output] + public string AndroidBinUtilsPath { get; set; } + public override bool Execute () { // OS X: $prefix/lib/xamarin.android/xbuild/Xamarin/Android @@ -68,6 +71,7 @@ public override bool Execute () } MonoAndroidBinPath = MonoAndroidHelper.GetOSBinPath () + Path.DirectorySeparatorChar; MonoAndroidLibPath = MonoAndroidHelper.GetOSLibPath () + Path.DirectorySeparatorChar; + AndroidBinUtilsPath = MonoAndroidBinPath + "ndk" + Path.DirectorySeparatorChar; MonoAndroidHelper.RefreshSupportedVersions (ReferenceAssemblyPaths); @@ -105,6 +109,7 @@ public override bool Execute () Log.LogDebugMessage ($" {nameof (JavaSdkPath)}: {JavaSdkPath}"); Log.LogDebugMessage ($" {nameof (MonoAndroidBinPath)}: {MonoAndroidBinPath}"); Log.LogDebugMessage ($" {nameof (MonoAndroidToolsPath)}: {MonoAndroidToolsPath}"); + Log.LogDebugMessage ($" {nameof (AndroidBinUtilsPath)}: {AndroidBinUtilsPath}"); //note: this task does not error out if it doesn't find all things. that's the job of the targets return !Log.HasLoggedErrors; diff --git a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets index 09d6d3b908f..9af68450fb8 100755 --- a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets +++ b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets @@ -814,6 +814,7 @@ Copyright (C) 2011-2012 Xamarin. All rights reserved. + @@ -2852,7 +2854,7 @@ because xbuild doesn't support framework reference assemblies. ObjectFiles="@(_NativeAssemblyTarget)" ApplicationSharedLibraries="@(_ApplicationSharedLibrary)" DebugBuild="$(AndroidIncludeDebugSymbols)" - AndroidSdkBuildToolsPath="$(AndroidSdkBuildToolsPath)" + AndroidBinUtilsDirectory="$(AndroidBinUtilsDirectory)" /> @@ -2933,7 +2935,7 @@ because xbuild doesn't support framework reference assemblies. Condition="'$(AotAssemblies)' == 'True'" AndroidAotMode="$(AndroidAotMode)" AndroidNdkDirectory="$(_AndroidNdkDirectory)" - ToolsDirectory="$(MonoAndroidBinDirectory)" + AndroidBinUtilsDirectory="$(AndroidBinUtilsDirectory)" AndroidApiLevel="$(_AndroidApiLevel)" ManifestFile="$(IntermediateOutputPath)android\AndroidManifest.xml" SupportedAbis="@(_BuildTargetAbis)"