Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Musl support in CI #1190

Open
UebelAndre opened this issue Jul 13, 2021 · 6 comments
Open

Musl support in CI #1190

UebelAndre opened this issue Jul 13, 2021 · 6 comments

Comments

@UebelAndre
Copy link
Contributor

Are there any available hosts that support musl? If so, how would I go about configuring a cc_toolchain for it, otherwise, I suppose this would be a feature request to enable CI tests for musl (related to bazelbuild/rules_rust#829).
@philwo
Copy link
Member

philwo commented Jul 13, 2021

@UebelAndre I never used musl - what would you need? Just apt install musl-dev in the Ubuntu 20.04 container or something more elaborate? 😀

@UebelAndre
Copy link
Contributor Author

I think gcc-multi and musl-dev? But the hard part would probably be configuring the cc_toolchain or at least in a way that didn't require users to have intimate knowledge of the base image. Perhaps an alpine image (which is musl by default) would do the trick?

@UebelAndre UebelAndre mentioned this issue Jul 15, 2021
Merged
@UebelAndre
Copy link
Contributor Author

Hi, any updates here? Is it possible to get an alpine image for use in CI?

@philwo
Copy link
Member

philwo commented Sep 6, 2021

@UebelAndre I'm not very experienced with Alpine Linux yet. I just tried this:

$ docker run --rm -it alpine:3.14
# mkdir test && cd test && touch WORKSPACE BUILD
# wget https://github.com/bazelbuild/bazel/releases/download/4.2.1/bazel-4.2.1-linux-x86_64
# chmod +x bazel-4.2.1-linux-x86_64
# ./bazel-4.2.1-linux-x86_64 info
/bin/sh: ./bazel-4.2.1-linux-x86_64: not found

Running ldd bazel-4.2.1-linux-x86_64 shows that a lot of symbols cannot be resolved, probably because we link against glibc:

# ldd bazel-4.2.1-linux-x86_64
	/lib64/ld-linux-x86-64.so.2 (0x7f8d35ca6000)
	librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7f8d35ca6000)
	libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f8d35ca6000)
	libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f8d35ca6000)
	libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f8d35ca6000)
Error loading shared library libstdc++.so.6: No such file or directory (needed by bazel-4.2.1-linux-x86_64)
Error loading shared library libgcc_s.so.1: No such file or directory (needed by bazel-4.2.1-linux-x86_64)
	libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f8d35ca6000)
Error relocating bazel-4.2.1-linux-x86_64: _ZdlPv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _Znwm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSsC1ERKSs: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs4swapERSs: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs4_Rep10_M_disposeERKSaIcE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _Unwind_Resume: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSsC1EPKcRKSaIcE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt18basic_stringstreamIcSt11char_traitsIcESaIcEED1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt17__throw_bad_allocv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt6localeD1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt25__throw_bad_function_callv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt8ios_baseD2Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs4_Rep10_M_destroyERKSaIcE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt29_Rb_tree_insert_and_rebalancebPSt18_Rb_tree_node_baseS0_RS_: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs4_Rep9_S_createEmmRKSaIcE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_begin_catch: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_rethrow: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_end_catch: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs7reserveEm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6appendEPKcm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6appendERKSs: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSsC1EPKcmRKSaIcE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6assignERKSs: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt19__throw_logic_errorPKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6insertEmPKcm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs7compareERKSs: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt18_Rb_tree_decrementPSt18_Rb_tree_node_base: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt20__throw_system_errori: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt20__throw_length_errorPKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt18basic_stringstreamIcSt11char_traitsIcESaIcEEC1ESt13_Ios_Openmode: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt16__ostream_insertIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_PKS3_l: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSt15basic_stringbufIcSt11char_traitsIcESaIcEE3strEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6insertEmPKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt18_Rb_tree_incrementPKSt18_Rb_tree_node_base: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt18_Rb_tree_incrementPSt18_Rb_tree_node_base: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs4findEcm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSsC1ERKSsmm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_guard_acquire: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_guard_release: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_guard_abort: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt20__throw_out_of_rangePKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSt8__detail20_Prime_rehash_policy14_M_need_rehashEmmm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt11_Hash_bytesPKvmm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt8ios_base4InitC1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt8ios_base4InitD1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt6chrono3_V212system_clock3nowEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs7compareEmmRKSs: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt11this_thread11__sleep_forENSt6chrono8durationIlSt5ratioILl1ELl1EEEENS1_IlS2_ILl1ELl1000000000EEEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSdD2Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6assignEPKcm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt6thread15_M_start_threadESt10shared_ptrINS_10_Impl_baseEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt6thread4joinEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt9terminatev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs7compareEPKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSt8__detail20_Prime_rehash_policy11_M_next_bktEm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_pure_virtual: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __gxx_personality_v0: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs12_M_leak_hardEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs4findEPKcmm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt8ios_baseC2Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt9basic_iosIcSt11char_traitsIcEE4initEPSt15basic_streambufIcS1_E: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt6localeC1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSolsEi: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs7compareEmmPKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6assignEPKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs12_S_constructEmcRKSaIcE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6resizeEmc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt13basic_filebufIcSt11char_traitsIcEEC1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt13basic_filebufIcSt11char_traitsIcEE4openEPKcSt13_Ios_Openmode: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt9basic_iosIcSt11char_traitsIcEE5clearESt12_Ios_Iostate: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSirsERi: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt13basic_filebufIcSt11char_traitsIcEE5closeEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt12__basic_fileIcED1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt13basic_filebufIcSt11char_traitsIcEED1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt14basic_ifstreamIcSt11char_traitsIcEED1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __realpath_chk: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSo5flushEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSolsEPSt15basic_streambufIcSt11char_traitsIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSo3putEc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSt5ctypeIcE13_M_widen_initEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt16__throw_bad_castv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt8__detail15_List_node_base7_M_hookEPS0_: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_allocate_exception: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt12bad_weak_ptrD1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_throw: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt28_Rb_tree_rebalance_for_erasePSt18_Rb_tree_node_baseRS_: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __memcpy_chk: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __fdelt_chk: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __strdup: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSsC1EmcRKSaIcE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt8__detail15_List_node_base9_M_unhookEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs9_M_mutateEmmm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _Znam: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZdaPv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __strncpy_chk: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt9exceptionD2Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __longjmp_chk: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt6thread20hardware_concurrencyEv: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt12out_of_rangeD1Ev: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __cxa_free_exception: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSt12out_of_rangeC1ERKSs: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: __once_proxy: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZnwmRKSt9nothrow_t: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZdlPvRKSt9nothrow_t: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6appendEPKc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs5rfindEcm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6appendEmc: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs7replaceEmmPKcm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs6appendERKSsmm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt18_Rb_tree_decrementPKSt18_Rb_tree_node_base: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNKSs12find_last_ofEPKcmm: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSo9_M_insertImEERSoT_: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSo9_M_insertIlEERSoT_: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt15__once_callable: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt11__once_call: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt9basic_iosIcSt11char_traitsIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt15basic_stringbufIcSt11char_traitsIcESaIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTISt12bad_weak_ptr: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVN10__cxxabiv117__class_type_infoE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVN10__cxxabiv121__vmi_class_type_infoE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt4cerr: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZSt7nothrow: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZNSs4_Rep20_S_empty_rep_storageE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVN10__cxxabiv120__si_class_type_infoE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTTSt19basic_ostringstreamIcSt11char_traitsIcESaIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTISt9exception: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt14basic_ifstreamIcSt11char_traitsIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt13basic_filebufIcSt11char_traitsIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt11logic_error: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt12out_of_range: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt18basic_stringstreamIcSt11char_traitsIcESaIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTTSt18basic_stringstreamIcSt11char_traitsIcESaIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt19basic_ostringstreamIcSt11char_traitsIcESaIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTTSt14basic_ifstreamIcSt11char_traitsIcEE: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt12bad_weak_ptr: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTISt12out_of_range: symbol not found
Error relocating bazel-4.2.1-linux-x86_64: _ZTVSt15basic_streambufIcSt11char_traitsIcEE: symbol not found

So it seems like this might not work. If you have an idea how we could make Bazel work under Alpine, I'd be happy to try it!

If it helps you, I'm tracking a feature request that allows projects on Bazel CI to install arbitrary packages using sudo apt install: #1214 - that way you could at least install musl-dev and gcc-multi. Maybe @hlopko could help us figure out how to create a cc_toolchain to use it?

@UebelAndre
Copy link
Contributor Author

I don't think you can use any of the released binaries from 4.2.1 since they were likely all compiled on glibc system. The solution will most likely involve bootstrapping or finding some resolution to bazelbuild/bazel#5891

I attempted the bootstrapping myself but am out of my element here

docker run --rm -it alpine:3.14
# Installing deps
apk add python3 bash openjdk11 openjdk8 gcc g++ musl-dev zip linux-headers
# Downloading Bazel
wget https://github.com/bazelbuild/bazel/releases/download/4.2.1/bazel-4.2.1-dist.zip
# Extracting it
mkdir bazel && unzip bazel-4.2.1-dist.zip -d bazel && cd bazel
# Bootstrapping
env EXTRA_BAZEL_ARGS="--host_javabase=@bazel_tools//tools/jdk:remote_jdk11 \
--javabase=@bazel_tools//tools/jdk:remote_jdk11 \
--host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java11 \
--java_toolchain=@bazel_tools//tools/jdk:toolchain_java11" bash ./compile.sh

This gets me as far as this error for which I have no solution or context for (been years since I worked with Java and never bootstrapped Bazel before).

ERROR: /tmp/bazel_XXEgBhoH/out/external/bazel_tools/tools/jdk/BUILD:346:14: Action external/bazel_tools/tools/jdk/platformclasspath_classes/DumpPlatformClassPath.class failed: (Exit -1): javac failed: error executing command
  (cd /tmp/bazel_XXEgBhoH/out/execroot/io_bazel && \
  exec env - \
  external/remotejdk11_linux/bin/javac -source 8 -target 8 -Xlint:-options -cp external/remotejdk11_linux/lib/tools.jar -d bazel-out/k8-opt/bin/external/bazel_tools/tools/jdk/platformclasspath_classes external/bazel_tools/tools/jdk/DumpPlatformClassPath.java)
Execution platform: //:default_host_platform
Action failed to execute: java.io.IOException: Cannot run program "/tmp/bazel_XXEgBhoH/out/execroot/io_bazel/external/remotejdk11_linux/bin/javac" (in directory "/tmp/bazel_XXEgBhoH/out/execroot/io_bazel"): error=2, No such file or directory
Target //src:bazel_nojdk failed to build

Does this shine light any a potential missing flag or dependency?

@UebelAndre
Copy link
Contributor Author

Friendly ping here.

@wenerme wenerme mentioned this issue Jan 13, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@philwo @UebelAndre