Make ‘cc_haskell_import’ work with ‘haskell_binary’ targets

[mrkkrp]

Close #179.

This PR allows to generate .so files from object files of executables, not just executables themselves. This way it's possible to use them with cc_haskell_import. Generation of .so version in conditional though, and disabled by default. The reasons:

• Some existing sh_tests run outputs of targets, and so if executable target produces more than one file they get confused. This could be avoided, if they just used executable from DefaultInfo, which always be a single file, but somehow this is not what sh_test does. We use our own copy of it, so we could patch it and submit the changes upstream.

• Usually such .so versions of binaries are not needed, so why do extra work.

The both reasons are not very strong IMO, so I'm open to alternative decisions.

[mboes]

As to whether we should create an extra output always or have a flag for that - let's do the former, not the latter.

Bazel only generates the outputs that are needed by dependencies. Listing the output in DefaultInfo only makes it a default output. So if the output is never requested downstream then we'll never do the work to create it, provided the rule actions are structured correctly (independent actions for independent outputs).

But I don't understand what this PR is doing. A Haskell binary is a binary, not a library. #179 did not call for sometimes making a Haskell binary a library, whether through an explicit flag or as an extra (non-default) output. Even when a binary is compiled with -pie, it's still a bona fide executable, named exactly as the user specified.

All I was expecting from this PR was that a provider be returned by haskell_binary, and that this provider be common to haskell_library as well (probably by decomposing the monolithic HaskellPackageInfo into smaller pieces as mentioned in the ticket), so that cc_haskell_import can uniformly deal with either, provided the binary was compiled as PIE (which incidentally, should probably be the default, or perhaps even always the case, given the security benefits).

[mrkkrp]

I tried what you describe (only generate binary) but I could not make C program link against such a binary exposed via cc_haskell_import. I was getting unresolved symbol errors.

Bazel only generates the outputs that are needed by dependencies. Listing the output in DefaultInfo only makes it a default output. So if the output is never requested downstream then we'll never do the work to create it, provided the rule actions are structured correctly (independent actions for independent outputs).

The main issue is not this, but that label of executable target no longer designates single file, but several and so it cannot be used in sh_test for example.

I also have the impression that only outputs listed in DefaultInfo force compilation, and that if you don't specify something is DefaultInfo at all, it won't be ever created. But I may be mistaken.

[mboes]

The main issue is not this, but that label of executable target no longer designates single file, but several and so it cannot be used in sh_test for example.

Yes. You mentioned in the PR description that an alternative would be to fix sh_test and then contribute upstream. I'm all for that. But I want to first understand why a .so is a necessity in the first place. It's possible that cc_* rules do something special based on the name of the output. In that case the only "work" necessary should be creating a symlink called libfoo.so to binary foo.

[mrkkrp]

I don't think this is going to work because while I was experimenting with this, I tried generating an executable and naming it with lib prefix and .so extension and it was not accepted by the linker. It started working only when I added the -shared flag, which means that we need two outputs: binary and so library if we want this to be visible to C code via cc_haskell_import.

[mrkkrp]

Can try once again tomorrow if you like.

[mrkkrp]

Amended the PR.

• The refactoring moves generation of dummy archive in its own function.

• We need to use dynamic object files if we want to compile with -dynamic.

• Handling of dynamic object files moved to _compilation_defaults so we have everything in one place (handling of dynamic and static object files). This will also allow to drop _hs_srcs and we'll be able to build list of source files (including source pre-processed with hsc2hs)/object files, etc. in one place only, in _compilation_defaults, and avoid re-building it. I did this in a later PR.

• Setting allow_single_file = True for dep attribute of cc_haskell_import won't work because rules like haskell_library generate multiple files (in its DefaultInfo), so Bazel will complain.

• According to the docs of cc_haskell_import we should make available

  the shared object files produced as well as the object files it depends on directly and transitively

  I preserved this logic, instead of making available only file from direct target (your branch). (I guess there is a reason it's done this way.)

So far I can't build clodl with Bazel so I can't test if it works with the changes.