Fix CMake Support and Enhance Newlib Targets #1112

Alexhuszagh · 2022-11-04T00:14:29Z

Fix CMake Support and Enhance Newlib Targets

Fix CMake support for Android targets by providing a toolchain file that specifies the API level and the standalone toolchain path, and ensuring that compiler detection is forcibly disabled. Fix CMake support for newlib targets by providing a toolchain file that signals to CMake it is compiling for an embedded target, providing correct C, C++, and ASM flags even when invoked externally, and ensuring CMake skips testing if the C and C++ compilers work by running a binary file. Also add C++ support to bare-metal newlib targets, by also installing libstdc++-arm-none-eabi-newlib. Finally, targets.toml was updated to ensure newlib targets do not have std support.

Other minor changes including using the Cortex-M1 CPU for thumbv6m-none-eabi rather than the Cortex-M3 (an ARMv7-M CPU) for the Qemu runner.

To ensure these work, test suites for CMake using C+11 features, CMake's try_run feature (checking CMAKE_CROSSCOMPILING_EMULATOR is set), and a no_std package with a C++ dependency have been added.

Closes #1110.
Closes #1113.

Alexhuszagh · 2022-11-04T00:14:52Z

Note that this isn't close to complete yet. This just adds the CMake unittests in its current form (to test which targets are failing).
bors try

Fixed/Changed:

CMake support for newlib targets
CMake support for ARM 32-bit Android targets
CMake support for ARM64 and x86/x86_64 targets
Add C++ support for newlib targets
Test C++ support on newlib targets.

I need to add a Changelog entry. We don't actually need to care about supporting Meson, since the only Meson build system for Rust as a crate is from 2017, and Meson itself tries to entirely recreate the Rust build system which is absolutely cursed. We would need to provide our own Meson cross-compilation config files for each target we'd want to cross-compile to since the meson crate does not support any cross-compilation automatically.

This is currently being blocked on a few other PRs:

Once all merge, I'll rebase, fix the THUMBv8 targets, and we should be ready.

bors · 2022-11-04T00:46:17Z

try

Build failed:

conclusion

Alexhuszagh · 2022-11-04T22:59:49Z

bors try --target android

bors · 2022-11-04T23:09:43Z

try

Build failed:

conclusion

Alexhuszagh · 2022-11-05T00:16:31Z

bors try --target android

bors · 2022-11-05T00:27:58Z

try

Build succeeded:

conclusion

Alexhuszagh · 2022-11-05T01:16:47Z

This currently uses a rust-cmake-hello-world which won't work for no_std targets, so we really should have both a library and a binary, where CMake module just does a no_std calculation or something.

Alexhuszagh · 2022-11-05T03:19:27Z

I've fixed CMake support for Android and newlib here, as well as added C++ support for newlib.

bors try --target android --target none

Alexhuszagh · 2022-11-05T03:19:53Z

ci/test.sh

@@ -71,8 +71,6 @@ main() {
        popd

        rm -rf "${td}"
-


We now want to run the CMake tests as well.

Alexhuszagh · 2022-11-05T03:20:11Z

ci/test.sh

@@ -166,7 +164,7 @@ main() {
    fi

    # Test C++ support
-    if (( ${CPP:-0} )); then
+    if (( ${STD:-0} )) && (( ${CPP:-0} )); then


This assumes we have C++ and std support, so we have to make that explicit.

Alexhuszagh · 2022-11-05T03:20:45Z

ci/test.sh

+
+    pushd "${td}"
+    retry cargo fetch
+    if (( ${STD:-0} )) && (( ${RUN:-0} )); then


The rust-cmake-hello-world has both a no_std lib and a std binary, so we can check support for either/or.

Alexhuszagh · 2022-11-05T03:22:35Z

docker/Dockerfile.thumbv6m-none-eabi

-    libnewlib-arm-none-eabi && \
-    /qemu.sh arm
+    libnewlib-arm-none-eabi \
+    libstdc++-arm-none-eabi-newlib


These targets actually support C++, and gcc-arm-none-eabi actually contains the C++ compiler, just not libstdc++v3. If we install that, these targets support C++ just fine. This applies for all newlib targets.

Alexhuszagh · 2022-11-05T03:23:42Z

docker/Dockerfile.thumbv6m-none-eabi


-ENV QEMU_CPU=cortex-m3 \
+ENV QEMU_CPU=cortex-m1 \


cortex-m3 is an ARMv7-M target, cortex-m1 is the proper CPU here for ARMv6-M (see ARM Cortex-M).

Alexhuszagh · 2022-11-05T03:24:21Z

docker/Dockerfile.thumbv6m-none-eabi

+
+COPY newlib.cmake /opt/newlib.cmake
+ENV CMAKE_TOOLCHAIN_FILE=/opt/newlib.cmake
+ENV CROSS_NEWLIB_OBJECT_FLAGS="-ffunction-sections -fdata-sections -mthumb -march=armv6s-m"


See newlib.cmake below for more info on why these defaults are provided. These flags are identical to the ones provided by cmake-rs, so they're used as the defaults to ensure anyone using cmake externally will have these same default flags. This applies for all newlib targets.

Alexhuszagh · 2022-11-05T03:29:36Z

docker/android.cmake

+# toolchain file for android targets, see #1110
+
+set(CMAKE_SYSTEM_NAME Android)
+set(CMAKE_SYSTEM_PROCESSOR "$ENV{CROSS_ANDROID_ARCH}")


This is not the same as the arch elsewhere: it must be i686, aarch64, x86_64, armv7-a, or armv5te. Just setting arm will not work.

Alexhuszagh · 2022-11-05T03:30:24Z

docker/android.cmake

+set(CMAKE_SYSTEM_NAME Android)
+set(CMAKE_SYSTEM_PROCESSOR "$ENV{CROSS_ANDROID_ARCH}")
+set(CMAKE_ANDROID_STANDALONE_TOOLCHAIN /android-ndk)
+set(CMAKE_ANDROID_API "$ENV{CROSS_ANDROID_SDK}")


We can't auto-detect the Android API, so we manually set it. CMake will still warn about this, but it's irrelevant since it's manually set.

Alexhuszagh · 2022-11-05T03:32:57Z

docker/android.cmake

+
+set(CMAKE_SYSTEM_NAME Android)
+set(CMAKE_SYSTEM_PROCESSOR "$ENV{CROSS_ANDROID_ARCH}")
+set(CMAKE_ANDROID_STANDALONE_TOOLCHAIN /android-ndk)


Since we use a standalone Android NDK with the /android-ndk/sysroot with no version prefix, we need to use CMAKE_ANDROID_STANDALONE_TOOLCHAIN and not CMAKE_SYSROOT nor CMAKE_ANDROID_NDK.

See the docs here.

Alexhuszagh · 2022-11-05T03:34:45Z

docker/android.cmake

+# not all of these are standard, however, they're common enough
+# that it's good practice to define them.
+set(prefix "$ENV{CROSS_ANDROID_TARGET}")
+set(CMAKE_C_COMPILER "${prefix}-gcc")


Only CMAKE_AR, CMAKE_RANLIB, CMAKE_C_COMPILER, CMAKE_ASM_COMPILER, and CMAKE_CXX_COMPILER are standard, however, other tools will often use values provided by those variables even though they're not standard. Many toolchain files will provide them.

It turns out setting these manually for the CMAKE_ASM_COMPILER, etc., fails badly with downstream configurations. See #1110 (comment).

Alexhuszagh · 2022-11-05T03:35:11Z

docker/android.cmake

+CMAKE_FORCE_C_COMPILER("${CMAKE_C_COMPILER}" GNU)
+CMAKE_FORCE_CXX_COMPILER("${CMAKE_CXX_COMPILER}" GNU)
+
+set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)


Standard for cross-compiling.

docker/newlib.cmake

Alexhuszagh · 2022-11-29T03:12:11Z

docker/Dockerfile.armv7-linux-androideabi

+    CROSS_CMAKE_SYSTEM_NAME=Android \
+    CROSS_CMAKE_SYSTEM_PROCESSOR=armv7-a \
+    CROSS_CMAKE_CRT=android \
+    CROSS_CMAKE_OBJECT_FLAGS="--target=arm-linux-androideabi -DANDROID -ffunction-sections -fdata-sections -fPIC --target=armv7-linux-androideabi"


I believe this change is correct, with --target provided twice, since that's what cmake-rs does as of 0.1.49 and cc-rs as of 1.0.77 (this is using Rust 1.64.0).

Alexhuszagh · 2022-11-29T04:28:45Z

bors try

bors · 2022-11-29T05:16:47Z

try

Build succeeded:

conclusion

Emilgardis

lgtm

Fixes CMake support for all targets, ensuring support for C11, C++11, `try_run`, and more features. To do so, we've included tests with a CMake test suite that compiles for a bare-metal target, one that compiles against [re2](https://github.com/google/re2), and one using CMake's `try_run` (checking [CMAKE_CROSSCOMPILING_EMULATOR](https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING_EMULATOR.html) is set for any cross-compiled targets). CMake support for Android targets was fixed by providing a toolchain file that specifies the API level and the standalone toolchain path, and ensuring that compiler detection is forcibly disabled. For other targets, a default toolchain file is provided that ensures the correct system name, processor, cross-compiling emulator, binutils, and cross-compilers are used. For bare-metal (newlib) targets, this ensures that `try_compile` uses a static library rather than an executable to avoid requiring a linker script to detect if the compiler works. This also adds a few minor changes for newlib targets: - Adds C++ support to bare-metal newlib targets by installing `libstdc++-arm-none-eabi-newlib`. - Updates `targets.toml` to specify that newlib targets do not have `std` support. - Use the Cortex-M1 CPU for `thumbv6m-none-eabi` rather than the Cortex-M3 (an ARMv7-M CPU) for the Qemu runner.

Alexhuszagh · 2022-12-04T15:27:11Z

bors r=Emilgardis

bors · 2022-12-04T16:20:21Z

Build succeeded:

conclusion

Alexhuszagh added enhancement container-images labels Nov 4, 2022

bors bot added a commit that referenced this pull request Nov 4, 2022

Try #1112:

659f424

Alexhuszagh force-pushed the cmake branch from 83bf87b to 6abdfec Compare November 4, 2022 22:57

bors bot added a commit that referenced this pull request Nov 4, 2022

Try #1112: --target *android*

54b782b

Alexhuszagh changed the title ~~Ensure CMake and Meson Build Support~~ Fix CMake Support and Test External Toolchain Support Nov 4, 2022

bors bot added a commit that referenced this pull request Nov 5, 2022

Try #1112: --target *android*

9d09856

Alexhuszagh mentioned this pull request Nov 5, 2022

Remove std = true for newlib targets. #1114

Closed

bors bot added a commit that referenced this pull request Nov 5, 2022

Try #1112: --target *android* --target *none*

8ac0bec

Alexhuszagh commented Nov 5, 2022

View reviewed changes

ci/test.sh

@@ -71,8 +71,6 @@ main() {

popd

rm -rf "${td}"

Copy link

Contributor Author

Alexhuszagh Nov 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now want to run the CMake tests as well.

Alexhuszagh commented Nov 5, 2022

View reviewed changes

docker/newlib.cmake Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

bors bot added a commit that referenced this pull request Nov 14, 2022

Try #1112:

4a588db

This comment was marked as outdated.

Sign in to view

Alexhuszagh force-pushed the cmake branch 3 times, most recently from 3002069 to 833b6ae Compare November 15, 2022 13:28

Alexhuszagh mentioned this pull request Nov 20, 2022

Add CROSS_TOOLCHAIN_PREFIX, Better CMake Support cross-rs/cross-toolchains#26

Open

Alexhuszagh force-pushed the cmake branch from 833b6ae to ec3a2e8 Compare November 29, 2022 02:25

Alexhuszagh marked this pull request as ready for review November 29, 2022 02:25

Alexhuszagh marked this pull request as draft November 29, 2022 02:27

Alexhuszagh force-pushed the cmake branch from ec3a2e8 to baae63d Compare November 29, 2022 03:09

Alexhuszagh commented Nov 29, 2022

View reviewed changes

This comment was marked as outdated.

Sign in to view

Alexhuszagh marked this pull request as ready for review November 29, 2022 03:18

bors bot added a commit that referenced this pull request Nov 29, 2022

Try #1112:

411d0a8

This comment was marked as outdated.

Sign in to view

bors bot added a commit that referenced this pull request Nov 29, 2022

Try #1112:

114f57f

Emilgardis self-requested a review November 29, 2022 08:27

Alexhuszagh mentioned this pull request Nov 29, 2022

Standard library headers not found by bindgen when cross-compiling for Windows #1147

Open

11 tasks

Emilgardis approved these changes Dec 4, 2022

View reviewed changes

Alexhuszagh force-pushed the cmake branch from da33f3b to cfbf581 Compare December 4, 2022 15:20

bors bot merged commit f0190b0 into cross-rs:main Dec 4, 2022

Alexhuszagh deleted the cmake branch December 4, 2022 19:43

Alexhuszagh mentioned this pull request Dec 5, 2022

Cannot build library with vendored zmq v0.9 for x86_64-linux-android #1167

Closed

Emilgardis added this to the v0.3.0 milestone Dec 29, 2022

Emilgardis mentioned this pull request May 29, 2023

Cannot build proj with newer images #1270

Open

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix CMake Support and Enhance Newlib Targets #1112

Fix CMake Support and Enhance Newlib Targets #1112

Alexhuszagh commented Nov 4, 2022 •

edited

Loading

Alexhuszagh commented Nov 4, 2022 •

edited

Loading

bors bot commented Nov 4, 2022

Alexhuszagh commented Nov 4, 2022

bors bot commented Nov 4, 2022

Alexhuszagh commented Nov 5, 2022

bors bot commented Nov 5, 2022

Alexhuszagh commented Nov 5, 2022

Alexhuszagh commented Nov 5, 2022

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 5, 2022 •

edited

Loading

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 5, 2022 •

edited

Loading

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 5, 2022

Alexhuszagh Nov 6, 2022

Alexhuszagh Nov 5, 2022

This comment was marked as outdated.

This comment was marked as outdated.

Alexhuszagh Nov 29, 2022

This comment was marked as outdated.

This comment was marked as outdated.

Alexhuszagh commented Nov 29, 2022

bors bot commented Nov 29, 2022

Emilgardis left a comment

Alexhuszagh commented Dec 4, 2022

bors bot commented Dec 4, 2022


		ENV QEMU_CPU=cortex-m3 \
		ENV QEMU_CPU=cortex-m1 \

Fix CMake Support and Enhance Newlib Targets #1112

Fix CMake Support and Enhance Newlib Targets #1112

Conversation

Alexhuszagh commented Nov 4, 2022 • edited Loading

Alexhuszagh commented Nov 4, 2022 • edited Loading

bors bot commented Nov 4, 2022

try

Alexhuszagh commented Nov 4, 2022

bors bot commented Nov 4, 2022

try

Alexhuszagh commented Nov 5, 2022

bors bot commented Nov 5, 2022

try

Alexhuszagh commented Nov 5, 2022

Alexhuszagh commented Nov 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Alexhuszagh Nov 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Alexhuszagh Nov 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as outdated.

This comment was marked as outdated.

Choose a reason for hiding this comment

This comment was marked as outdated.

This comment was marked as outdated.

Alexhuszagh commented Nov 29, 2022

bors bot commented Nov 29, 2022

try

Emilgardis left a comment

Choose a reason for hiding this comment

Alexhuszagh commented Dec 4, 2022

bors bot commented Dec 4, 2022

Alexhuszagh commented Nov 4, 2022 •

edited

Loading

Alexhuszagh commented Nov 4, 2022 •

edited

Loading

Alexhuszagh Nov 5, 2022 •

edited

Loading

Alexhuszagh Nov 5, 2022 •

edited

Loading