cabal new-build fails on time library

[RyanGlScott]

I noticed this when debugging haskell/deepseq#39. To reproduce:

$ cabal get time-1.9.1
Unpacking to time-1.9.1/
$ cd time-1.9.1/
$ cabal new-build
Resolving dependencies...
Build profile: -w ghc-8.2.2 -O1
In order, the following will be built (use -v for more details):
 - time-1.9.1 (lib:time) (first run)
Configuring time-1.9.1...
checking for gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /usr/bin/gcc accepts -g... yes
checking for /usr/bin/gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... /usr/bin/gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking time.h usability... yes
checking time.h presence... yes
checking for time.h... yes
checking for gmtime_r... yes
checking for localtime_r... yes
checking for clock_gettime... yes
checking whether struct tm is in sys/time.h or time.h... time.h
checking for struct tm.tm_zone... yes
checking whether time.h and sys/time.h may both be included... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking whether altzone is declared... no
configure: creating ./config.status
config.status: creating lib/include/HsTimeConfig.h
Preprocessing library for time-1.9.1..
Building library for time-1.9.1..
<build output elided>
cabal: can't find include file HsTimeConfig.h

$ cabal --version
cabal-install version 2.2.0.0
compiled using version 2.2.0.0 of the Cabal library

On the other hand, cabal build works without issue. This is because the generated HsTimeConfig.h file seems to be put in a place where cabal can't find it:

$ find . -iname "HsTimeConfig.h"
./dist-newstyle/build/x86_64-linux/ghc-8.2.2/time-1.9.1/build/lib/include/HsTimeConfig.h

[phadej]

Hmm, this looks like #5169 but in this time case, the include-dirs: lib/include is there.

[phadej]

ping @angerman @hvr

[RyanGlScott]

Ah yes, good catch. I think this is indeed a duplicate of #5169, since new-build works if I switch to cabal-install-2.0.

I'll close this in favor of #5169.

[hvr]

I'm reopening this, as this one's different enough from #5169, insofar as I consider this one a fixable issue; quoting #5169 (comment)

This is about about install-includes referring to an auto-generated header file which lib:Cabal is trying to locate (but looks in the wrong place; ideally we'd have the equivalent of autogen-modules for include header files, then cabal would know the single place where to expect that file, rather than have to try two places), rather than an -I flag not being passed to GHC (and GHC complaining).

This was pointed out by @phadej as well here; so what we could do is have Cabal look also in the autogen folder for (relative) folders explicitly specified as include-dirs.

@angerman what do you think?

[angerman]

@hvr guess that could work. I'm a bit weary if we end up looking for headers just all over the place. I'm not sure if we could (or even should) try to enforce a better system for generated files.

[RyanGlScott]

This also prevents libraries like process from building against a new-built unix:

Configuring process-1.6.3.0...
configure: WARNING: unrecognized options: --with-compiler
checking for gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /usr/bin/gcc accepts -g... yes
checking for /usr/bin/gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... /usr/bin/gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for pid_t... yes
checking vfork.h usability... no
checking vfork.h presence... no
checking for vfork.h... no
checking for fork... yes
checking for vfork... yes
checking for working fork... yes
checking for working vfork... (cached) yes
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking sys/wait.h usability... yes
checking sys/wait.h presence... yes
checking for sys/wait.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking for setitimer... yes
checking for sysconf... yes
checking value of SIG_DFL... -1
checking value of SIG_IGN... -1
configure: creating ./config.status
config.status: creating include/HsProcessConfig.h
configure: WARNING: unrecognized options: --with-compiler
Preprocessing library for process-1.6.3.0..
Building library for process-1.6.3.0..
[1 of 5] Compiling System.Process.Common ( System/Process/Common.hs, /home/rgscott/Software/unix/dist-newstyle/build/x86_64-linux/ghc-8.4.1/process-1.6.3.0/build/System/Process/Common.o )
[2 of 5] Compiling System.Process.Posix ( System/Process/Posix.hs, /home/rgscott/Software/unix/dist-newstyle/build/x86_64-linux/ghc-8.4.1/process-1.6.3.0/build/System/Process/Posix.o )
[3 of 5] Compiling System.Process.Internals ( System/Process/Internals.hs, /home/rgscott/Software/unix/dist-newstyle/build/x86_64-linux/ghc-8.4.1/process-1.6.3.0/build/System/Process/Internals.o )
[4 of 5] Compiling System.Process   ( System/Process.hs, /home/rgscott/Software/unix/dist-newstyle/build/x86_64-linux/ghc-8.4.1/process-1.6.3.0/build/System/Process.o )
[5 of 5] Compiling System.Cmd       ( System/Cmd.hs, /home/rgscott/Software/unix/dist-newstyle/build/x86_64-linux/ghc-8.4.1/process-1.6.3.0/build/System/Cmd.o )

In file included from cbits/runProcess.c:19:0: error: 

/home/rgscott/Software/unix/include/execvpe.h:15:26: error:
     fatal error: HsUnixConfig.h: No such file or directory
     #include "HsUnixConfig.h"
                              ^
   |
15 | #include "HsUnixConfig.h"
   |                          ^
compilation terminated.
`gcc' failed in phase `C Compiler'. (Exit code: 1)

[quasicomputational]

This seems like something we should try to sort out before new-style commands become the default. cc @typedrat.

[hvr]

@RyanGlScott does this only occur when process or time are inplace or local packages?

I.e. this doesn't appear to be an issue for non-inplace cases as in e.g.

cabal new-repl -w ghc-8.2.2 --build-d 'time == 1.9.1'

[RyanGlScott]

@hvr, correct. I only experienced these errors when trying to get these packages in-place.

The original context in which I discovered this bug was haskell/deepseq#39, where I needed to build time in-place since it depends on deepseq, which I also had to build in-place. But one can more directly reproduce the bug by simply running cabal get time and then building that in-place.

[arrowd]

It seems, I got the same problem when building git-annex.

I've put all its dependencies, including network and network-multicast into source dir and then ran cabal new-build. It failed with

Configuring network-multicast-0.2.0...
Preprocessing library for network-multicast-0.2.0..
In file included from Multicast.hsc:15:
In file included from /wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/network-2.7.0.2/include/HsNet.h:10:
/wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/network-2.7.0.2/include/HsNetDef.h:4:10: fatal error: 'HsNetworkConfig.h' file not found
#include "HsNetworkConfig.h"
         ^~~~~~~~~~~~~~~~~~~
1 error generated.
compiling /wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/dist-newstyle/build/x86_64-freebsd/ghc-8.6.3/network-multicast-0.2.0/build/Network/Multicast_hsc_make.c failed (exit code 1)
command was: /usr/bin/cc -c /wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/dist-newstyle/build/x86_64-freebsd/ghc-8.6.3/network-multicast-0.2.0/build/Network/Multicast_hsc_make.c -o /wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/dist-newstyle/build/x86_64-freebsd/ghc-8.6.3/network-multicast-0.2.0/build/Network/Multicast_hsc_make.o -fno-stack-protector -fno-stack-protector -D__GLASGOW_HASKELL__=806 -Dfreebsd_BUILD_OS=1 -Dx86_64_BUILD_ARCH=1 -Dfreebsd_HOST_OS=1 -Dx86_64_HOST_ARCH=1 -I/wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/dist-newstyle/build/x86_64-freebsd/ghc-8.6.3/network-multicast-0.2.0/build/autogen -I/wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/dist-newstyle/build/x86_64-freebsd/ghc-8.6.3/network-multicast-0.2.0/build/global-autogen -include /wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/dist-newstyle/build/x86_64-freebsd/ghc-8.6.3/network-multicast-0.2.0/build/autogen/cabal_macros.h -I/wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/network-2.7.0.2/include -I/usr/local/lib/ghc-8.6.3/unix-2.7.2.2/include -I/usr/local/lib/ghc-8.6.3/time-1.8.0.2/include -I/usr/local/lib/ghc-8.6.3/bytestring-0.10.8.2/include -I/usr/local/include -I/usr/local/lib/ghc-8.6.3/base-4.12.0.0/include -I/usr/local/include -I/usr/local/lib/ghc-8.6.3/integer-gmp-1.0.2.0/include -I/usr/local/lib/ghc-8.6.3/include -I/usr/local/include -I/usr/local/lib/ghc-8.6.3/include/
cabal: Failed to build network-multicast-0.2.0 (which is required by git-annex-7.20190129).

The /wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/network-2.7.0.2/include directory doesn't contain HsNetworkConfig.h file, but does have HsNetworkConfig.h.in one.
At the same time, /wrkdirs/usr/ports/devel/hs-git-annex/work/git-annex-7.20190129/dist-newstyle/build/x86_64-freebsd/ghc-8.6.3/network-2.7.0.2/build/include/ does contain required header, but there is no -I flag with that dir.

[phadej]

This still happens for local time package.

Which means that we cannot even consider using haskell-ci for time.

[phadej]

This seems to be duplicate of #5169

[phadej]

My PR fixes the original time issue (which is sdist going weird).

However, unix + process is different issue; and easily reproducible as well. The problem is in what gets registed in the project local package-db:

data-dir: /code/mess/foo/unix
hs-libraries: HSunix-2.7.2.2-inplace
extra-libraries: rt util dl pthread
include-dirs: /code/mess/foo/unix/include  <----
includes: HsUnix.h execvpe.h
depends: base-4.12.0.0 bytestring-0.10.8.2 time-1.8.0.2
haddock-interfaces:
/code/mess/foo/dist-newstyle/build/x86_64-linux/ghc-8.6.5/unix-2.7.2.2/doc/html/unix/unix.haddock

The include-dirs is only half right: it does contain bunded headers, but not generated

% ls /code/mess/foo/unix/include
execvpe.h  HsUnixConfig.h.in  HsUnix.h

% find . -name 'HsUnixConfig.h' 
./dist-newstyle/build/x86_64-linux/ghc-8.6.5/unix-2.7.2.2/build/include/HsUnixConfig.h

so it should be

include-dirs: /code/mess/foo/unix/include /code/mess/foo/dist-newstyle/build/x86_64-linux/ghc-8.6.5/unix-2.7.2.2/build/

[chrissound]

I'm not sure if it's related but I seem to possibly running into this issue as well. I'm not sure what I could do to investigate further..

Essentially I have a cabal dependency on time >= 1.9.2 - this seems to fail with Nix as well?

cabal new-build
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: platinumpitanga-1.0.0.0 (user goal)
[__1] next goal: turtle (dependency of platinumpitanga)
[__1] rejecting: turtle-1.5.13/installed-1Sp... (conflict: platinumpitanga =>
time>=1.9.2, turtle => time==1.8.0.2/installed-1.8...)
[__1] trying: turtle-1.5.14
[__2] next goal: time (dependency of platinumpitanga)
[__2] rejecting: time-1.8.0.2/installed-1.8... (conflict: platinumpitanga =>
time>=1.9.2)
[__2] rejecting: time-1.9.3, time-1.9.2 (conflict: turtle => time<1.9)
[__2] rejecting: time-1.9.1, time-1.9, time-1.8.0.4, time-1.8.0.3,
time-1.8.0.2, time-1.8.0.1, time-1.8, time-1.7.0.1, time-1.7, time-1.6.0.1,
time-1.6, time-1.5.0.1, time-1.5, time-1.4.2, time-1.4.1, time-1.4.0.2,
time-1.4.0.1, time-1.4, time-1.3, time-1.2.0.5, time-1.2.0.4, time-1.2.0.3,
time-1.2.0.2, time-1.2.0.1, time-1.2, time-1.1.4, time-1.1.3, time-1.1.2.4,
time-1.1.2.3, time-1.1.2.2, time-1.1.2.1, time-1.1.2.0, time-1.0 (conflict:
platinumpitanga => time>=1.9.2)
[__2] fail (backjumping, conflict set: platinumpitanga, time, turtle)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: time, platinumpitanga, turtle