extra-source-files not visible

[aherrmann]

I'm encountering a build failure when trying to use cabal2bazel on a Haskell library that makes use of extra-source-files and CPP #include directives.

For example, assume the following source files exist: srcs/Some/Module.hs, includes/Some.Module.hs. And that srcs/Some/Module.hs contains:

{-# LANGUAGE CPP #-}
module Some.Module where
...
#include "../../includes/Some.Module.hs

And the cabal file contains:

extra-source-files:
  includes/Some.Module.hs

Then the cabal2bazel build fails with an error of the form

error: fatal error: ../../includes/Some.Module.hs: No such file or directory

The only reference to extra-source-files I could find is here, where it is used to construct -I command-line flags to GHC. It looks like, contrary to other source files, extra-source-files are not linked or copied into the working directory for builds. And even if they were, the include path would still be broken because cabal2bazel arranges source files relative to their hs-source-dirs entry and not to their position in the original source tree.

What is the reason for arranging source files relative to their hs-source-dirs entry instead of to their original position in the source tree? Couldn't the same effect be achieved by keeping files in their original source tree position and passing additional -i arguments to GHC?

This would allow to fix the above issue by simply linking or copying the extra-source-files into the build tree.

[judah]

This proposal sounds reasonable to me. I recall Formation also had one or two packages that we had to make manual overrides for due to this issue.

The reason all the files are arranged in one directory is that rules_haskell used to not allow a single rule having module files in different directories. The files needed a common prefix (the src_strip_prefix attribute) so that rules_haskell could translate filepaths into module names. However, that restriction was removed in tweag/rules_haskell#383, so the previous organization does not seem necessary anymore.

[aherrmann]

For a concrete example, the Hackage package uniplate triggers this issue.

$ bazel build @haskell_uniplate//:all
...
INFO: Analysed 29 targets (33 packages loaded).
INFO: Found 29 targets...
ERROR: /home/aj/.cache/bazel/_bazel_aj/5de16b08c1a0249c96a7b741e2682178/external/haskell_uniplate/BUILD:17:1: error executing shell command: '/nix/store/nyvnjw9vhmkba4a6wshy9404fx0rb24w-bash/bin/bash -c ${1+"$@"}  bazel-out/host/bin/external/ghc/visible-binaries/ghc -pgma bazel-out/host/bin/external/ghc/visible-binaries/cc -pgmc bazel-ou...' failed (Exit 1)

bazel-out/k8-fastbuild/bin/external/haskell_uniplate/gen-srcs-uniplate/Data/Generics/Uniplate/Data.hs:45:2: error:
     fatal error: Internal/DataInc.hs: No such file or directory
compilation terminated.
`cc' failed in phase `C pre-processor'. (Exit code: 1)
INFO: Elapsed time: 44.194s, Critical Path: 32.29s
INFO: 142 processes: 142 processwrapper-sandbox.
FAILED: Build did NOT complete successfully

[mjrussell]

I think I just hit this exact issue, will give your PR a try @aherrmann

[mjrussell]

Ah unfortunately Im running into issues building wai-app-static (some discussion here - tweag/rules_haskell#291 (comment)) and I was hoping your changes with the symlinks would also work for TH. It didn't appear to, although its not totally clear to me why.

Also, it seems like wai-app-static no longer works even with the forking due to the name mangling (i.e. looking for external/haskell_wai_app_static__1453541560/images/folder.png instead of external/haskell_wai_app_static/images/folder.png