rules_haskell is incompatible with Bazel's new "trim_test_configuration" default setting

[fweikert]

Describe the bug
Using a Bazel binary built at commit bazelbuild/bazel@ebac27e or later will crash certain rules_haskell targets with the following error:

ERROR: /usr/local/google/home/fwe/git/rules_haskell/tests/binary-with-sysdeps/BUILD.bazel:8:13: in _haskell_test rule //tests/binary-with-sysdeps:binary-with-sysdeps: 
Traceback (most recent call last):
	File "/usr/local/google/home/fwe/git/rules_haskell/haskell/private/haskell_impl.bzl", line 77, column 39, in haskell_test_impl
		return _haskell_binary_common_impl(ctx, is_test = True)
	File "/usr/local/google/home/fwe/git/rules_haskell/haskell/private/haskell_impl.bzl", line 173, column 25, in _haskell_binary_common_impl
		cc = cc_interop_info(ctx)
	File "/usr/local/google/home/fwe/git/rules_haskell/haskell/cc.bzl", line 166, column 40, in cc_interop_info
		cc_libraries = get_cc_libraries(cc_libraries_info, [lib for li in cc_common.merge_cc_infos(cc_infos = ccs).linking_context.linker_inputs.to_list() for lib in li.libraries]),
	File "/usr/local/google/home/fwe/git/rules_haskell/haskell/private/cc_libraries.bzl", line 49, column 43, in get_cc_libraries
		if not cc_libraries_info.libraries[cc_library_key(lib_to_link)].is_haskell
Error: key struct(dynamic_library = <generated file _solib_k8/[email protected]_S_S_Czlib___Uexternal_Szlib.hs/libz.so>, interface_library = None, pic_static_library = None, static_library = <generated file external/zlib.hs/libz.a>) not found in dictionary
ERROR: Analysis of target '//tests/binary-with-sysdeps:binary-with-sysdeps' failed; build aborted: Analysis of target '//tests/binary-with-sysdeps:binary-with-sysdeps' failed
INFO: Elapsed time: 7.251s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (71 packages loaded, 9505 targets configured)

To Reproduce

1. Download Bazelisk from https://github.com/bazelbuild/bazelisk
2. git clone rules_haskell repository
3. From within the repository run USE_BAZEL_VERSION="last_green" bazelisk build //tests/binary-with-sysdeps:binary-with-sysdeps

Expected behavior
Build succeeds

Environment

• OS name + version: Ubuntu 18.04
• Bazel version: any that contains bazelbuild/bazel@ebac27e
• Version of the rules: a757876

Additional context

• First observed on Bazel CI: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1990#4a984f89-939e-496c-bc0c-024daa6c059f
• This breakage only seems to affect some test targets, and it can be avoided by passing --notrim_test_configuration to Bazel (proof: Bazel CI: Downstream projects broken due to the flip of trim_test_configuration bazelbuild/bazel#13328 (comment)).
• As pointed out in Bazel CI: Downstream projects broken due to the flip of trim_test_configuration bazelbuild/bazel#13328 (comment) we would expect a "conflicting actions" error from Bazel, not a failure in a bzl file.

cc @sdtwigg

[fweikert]

There's some disagreement on whether this constitutes a Bazel bug or a problem in rules_haskell that was unearthed by the recent changes.

[aherrmann]

@fweikert Thanks for raising this and for repro.

I reran the repro with the following debugging code in place:

diff --git a/haskell/private/cc_libraries.bzl b/haskell/private/cc_libraries.bzl
index 3ac12fef..2b36ca9c 100644
--- a/haskell/private/cc_libraries.bzl
+++ b/haskell/private/cc_libraries.bzl
@@ -43,6 +43,7 @@ def get_cc_libraries(cc_libraries_info, libraries_to_link):
       list of LibraryToLink, C libraries.

     """
+    print("!!! get_cc_libraries", "".join(["\n  {}".format(k) for k in cc_libraries_info.libraries.keys()]))
     return [
         lib_to_link
         for lib_to_link in libraries_to_link

The output is

DEBUG: .../haskell/private/cc_libraries.bzl:46:10: !!! get_cc_libraries
  struct(dynamic_library = <generated file _solib_k8/[email protected]_S_S_Czlib___Uexternal_Szlib.hs/libz.so>, interface_library = None, pic_static_library = None, static_library = <generated file external/zlib.hs/libz.a>)
  struct(dynamic_library = <generated file _solib_k8/external_Srules_Uhaskell_Ughc_Ulinux_Uamd64/libHSbase-4.13.0.0-ghc8.8.3.so>, interface_library = None, pic_static_library = None, static_library = <source file lib/base-4.13.0.0/libHSbase-4.13.0.0.a>)
  struct(dynamic_library = <generated file _solib_k8/external_Srules_Uhaskell_Ughc_Ulinux_Uamd64/libHSinteger-gmp-1.0.2.0-ghc8.8.3.so>, interface_library = None, pic_static_library = None, static_library = <source file lib/integer-gmp-1.0.2.0/libHSinteger-gmp-1.0.2.0.a>)
  struct(dynamic_library = <generated file _solib_k8/external_Srules_Uhaskell_Ughc_Ulinux_Uamd64/libHSghc-prim-0.5.3-ghc8.8.3.so>, interface_library = None, pic_static_library = None, static_library = <source file lib/ghc-prim-0.5.3/libHSghc-prim-0.5.3.a>)
  struct(dynamic_library = <generated file _solib_k8/external_Srules_Uhaskell_Ughc_Ulinux_Uamd64/libHSrts_thr-ghc8.8.3.so>, interface_library = None, pic_static_library = None, static_library = <source file lib/rts/libHSrts_thr.a>)
  struct(dynamic_library = <generated file _solib_k8/external_Srules_Uhaskell_Ughc_Ulinux_Uamd64/libffi.so.7>, interface_library = None, pic_static_library = None, static_library = <source file lib/rts/libCffi_thr.a>)
Error: key struct(dynamic_library = <generated file _solib_k8/[email protected]_S_S_Czlib___Uexternal_Szlib.hs/libz.so>, interface_library = None, pic_static_library = None, static_library = <generated file external/zlib.hs/libz.a>) not found in dictionary

The failing code uses a struct over None or File as dictionary keys. The error message indicates that the looked up key is missing, however, the debug output suggests that it is actually present. I suspect that using File as dictionary keys is the issue here and something about --trim_test_configuration changes the identity of File between inserting them into the dictionary and looking them up. @fweikert does that make sense?

For context, the whole thing is part of a compatibility layer between Bazel's way of handling C libraries and what the Haskell compiler GHC expects. See #1241 and #1256 for details. In short, we generate symlinks for static libraries in an aspect to replicate Bazel's mangling of dynamic libraries, because GHC expects static and dynamic libraries to have the same name. We then keep a mapping from original LibraryToLink to a version with the mangled symlinks for later use.

Perhaps using File in a dictionary key is not such a good idea. It seems that a possible workaround for this issue is to use the File's path attribute instead.

[fweikert]

cc @sdtwigg as expert