Mostly static binaries

[mboes]

We already support building static libraries. In fact we already know how to static binaries as well: when profiling is enabled we use static linking because profiling and dynamic linking don't mix. This ticket is about exposing to the user the functionality, so that users can force static linking.

This ticket is about static linking in the sense of GHC: in Bazel terms, this corresponds to "mostly static" linking. That is, Haskell code is statically linked, but system libraries (like libgmp, glibc, libpthread etc) are still dynamically linked.

cc @lunaris

[lunaris]

I will try and get a patch going for this, because simply uncommenting the elif not with_profiling part of haskell_link_binary that adds -pie-etc. arguments seems to work just fine:

➜  test git:(bazel) ✗ ldd bazel-bin/engine/bin/static-binary/static-binary                                    
        linux-vdso.so.1 =>  (0x00007ffc9999f000)
        libstdc++.so.6 => /nix/store/zk5zj2307zxaq7dx585yia3dn5k4qlsl-gcc-7.3.0-lib/lib/libstdc++.so.6 (0x00007f84f5e39000)
        libm.so.6 => /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libm.so.6 (0x00007f84f5aa4000)
        libssh2.so.1 => /nix/store/n6i3f9q2r1ikbdhy6j51c30fslrhqvww-libssh2-1.8.0/lib/libssh2.so.1 (0x00007f84f5876000)
        libpcre.so.1 => /nix/store/3jg1sj3va3rmm4fhw5xkybq9q3v8wnyi-pcre-8.42/lib/libpcre.so.1 (0x00007f84f5604000)
        liblua.so.5.3 => /nix/store/m7bphi4y05byj5kqiafgqx19y6biv4wb-lua-5.3.5/lib/liblua.so.5.3 (0x00007f84f53cb000)
        libz.so.1 => /nix/store/bv6znzsv2qkbcwwa251dx7n5dshz3nr3-zlib-1.2.11/lib/libz.so.1 (0x00007f84f51af000)
        libpthread.so.0 => /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0 (0x00007f84f4f90000)
        libpq.so.5 => /nix/store/1206craj2bvn18slflp125mkh8ncp5dz-postgresql-9.6.10-lib/lib/libpq.so.5 (0x00007f84f4d61000)
        librt.so.1 => /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/librt.so.1 (0x00007f84f4b59000)
        libutil.so.1 => /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libutil.so.1 (0x00007f84f4956000)
        libdl.so.2 => /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libdl.so.2 (0x00007f84f4752000)
        libgmp.so.10 => /nix/store/dsjjfh28bqg78773r7zvq33giwsbyiqb-gmp-6.1.2/lib/libgmp.so.10 (0x00007f84f44be000)
        libgcc_s.so.1 => /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libgcc_s.so.1 (0x00007f84f42a8000)
        libc.so.6 => /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6 (0x00007f84f3ef4000)
        /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f84f61c0000)
        libssl.so.1.0.0 => /nix/store/gsgdj34vv5hh3iyyfm7yh69j6dq0br9q-openssl-1.0.2p/lib/libssl.so.1.0.0 (0x00007f84f3c80000)
        libcrypto.so.1.0.0 => /nix/store/gsgdj34vv5hh3iyyfm7yh69j6dq0br9q-openssl-1.0.2p/lib/libcrypto.so.1.0.0 (0x00007f84f3819000)

Fully-static however seems like it will be much more problematic -- adding the traditional -optl-static -optl-pthread to compiler_flags for a given haskell_binary produces lots of errors like:

...
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(guard.o):function __cxa_guard_abort: error: undefined reference to 'syscall'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(guard.o):function __cxa_guard_release: error: undefined reference to 'syscall'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(vterminate.o):function __gnu_cxx::__verbose_terminate_handler(): error: undefined reference to 'fputs'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(vterminate.o):function __gnu_cxx::__verbose_terminate_handler(): error: undefined reference to 'fputs'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(cp-demangle.o):function d_append_num: error: undefined reference to 'sprintf'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(cp-demangle.o):function d_print_comp_inner: error: undefined reference to 'sprintf'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(cp-demangle.o):function d_print_comp_inner: error: undefined reference to 'sprintf'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/../../../../lib64/libstdc++.a(cp-demangle.o):function d_print_comp_inner: error: undefined reference to 'sprintf'
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/libgcc_eh.a(unwind-dw2-fde-dip.o):function _Unwind_Find_FDE: error: undefined reference to 'dl_iterate_phdr'
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)
Target //engine/bin/static-binary:static-binary failed to build
INFO: Elapsed time: 30.358s, Critical Path: 29.15s
INFO: 1 process: 1 linux-sandbox.

[Profpatsch]

I get an error when I try to update to ghc-8.4.3:

ERROR: /home/philip/kot/bazel/rules_haskell/tests/binary-mostly-static/BUILD:8:1: error executing shell command: '/nix/store/wp2zql6995k7vlxhpjzaifzn2lcll8ql-bash/bin/bash -c ${1+"$@"} $(< bazel-out/k8-fastbuild/bin/tests/binary-mostly-static/_obj/binary-mostly-static.manifest)  bazel-out/host/bin/tests/visibl...' failed (Exit 1)
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(Capability.thr_o):Capability.c:function tryGrabCapability: error: undefined reference to 'pthread_mutex_trylock'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(Schedule.thr_o):Schedule.c:function schedule: error: undefined reference to 'pthread_mutex_trylock'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(OSThreads.thr_o):OSThreads.c:function createOSThread: error: undefined reference to 'pthread_create'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(OSThreads.thr_o):OSThreads.c:function createOSThread: error: undefined reference to 'pthread_detach'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(OSThreads.thr_o):OSThreads.c:function createOSThread: error: undefined reference to 'pthread_setname_np'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(OSThreads.thr_o):OSThreads.c:function interruptOSThread: error: undefined reference to 'pthread_kill'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(Itimer.thr_o):Itimer.c:function initTicker: error: undefined reference to 'pthread_create'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(Itimer.thr_o):Itimer.c:function initTicker: error: undefined reference to 'pthread_setname_np'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(Itimer.thr_o):Itimer.c:function exitTicker: error: undefined reference to 'pthread_join'
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3/lib/ghc-8.4.3/rts/libHSrts_thr.a(Itimer.thr_o):Itimer.c:function exitTicker: error: undefined reference to 'pthread_detach'
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

8.2.2 works fine. Will ping you on the PR.

Update: whoops, should have been on the PR.