How to [new-]build with Cabal 3?

[Bodigrim]

I cannot build time with cabal build, which defaults to new-build in recent Cabal:

git clone https://github.com/haskell/time.git
cd time
autoreconf -i
cabal build

Resolving dependencies...
Build profile: -w ghc-8.10.2 -O1
In order, the following will be built (use -v for more details):
 - time-1.11.1.1 (lib:time) (first run)
Configuring time-1.11.1.1...
<...>
configure: creating ./config.status
config.status: creating lib/include/HsTimeConfig.h
Preprocessing library for time-1.11.1.1..
Building library for time-1.11.1.1..
...
[46 of 46] Compiling Data.Time.Format.ISO8601 ( lib/Data/Time/Format/ISO8601.hs, /private/tmp/time/dist-newstyle/build/x86_64-osx/ghc-8.10.2/time-1.11.1.1/build/Data/Time/Format/ISO8601.o, /private/tmp/time/dist-newstyle/build/x86_64-osx/ghc-8.10.2/time-1.11.1.1/build/Data/Time/Format/ISO8601.dyn_o )
cabal: can't find include file HsTimeConfig.h

$ cabal --version
cabal-install version 3.4.0.0
compiled using version 3.4.0.0 of the Cabal library
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.10.2

However, cabal v1-build still works fine. The issues seems to be similar to haskell/cabal#5223, but that one is closed, so I'm a bit confused.

[AshleyYakeley]

This must be a defect in cabal, right?

[AshleyYakeley]

Can you build earlier versions of time in this manner? haskell/cabal#5223 is for time-1.9.1.

[AshleyYakeley]

In any case, you should be getting time from Hackage, not from git.

[Bodigrim]

Can you build earlier versions of time in this manner?
In any case, you should be getting time from Hackage, not from git.

I checked all versions starting from 1.9 (both from Git tags and from Hackage), cabal new-build consistently fails with the same error.

This must be a defect in cabal, right?

To be honest, I'm not well acquainted with time.cabal to judge. I decided that in any case it is worth bringing to your attention: even if this is a cabal defect indeed, could there possibly exist a reasonable workaround to apply on time side? I hit this issue on CI, and resorting to stack or cabal v1-build is not as straightforward as in local environment.

[AshleyYakeley]

If cabal fails with time-1.9.1, isn't this exactly the same as haskell/cabal#5223?

[Bodigrim]

That’s why I’m confused: haskell/cabal#5223 seems to be closed as resolved with regards to time. CC @phadej

[phadej]

time should use autogen-includes field.

diff --git a/time.cabal b/time.cabal
index ceb4b14..c977653 100644
--- a/time.cabal
+++ b/time.cabal
@@ -1,7 +1,8 @@
+cabal-version:  3.0
 name:           time
 version:        1.11.1.1
 stability:      stable
-license:        BSD3
+license:        BSD-3-Clause
 license-file:   LICENSE
 author:         Ashley Yakeley
 maintainer:     <[email protected]>
@@ -11,7 +12,6 @@ synopsis:       A time library
 description:    Time, clocks and calendars
 category:       Time
 build-type:     Configure
-cabal-version:  >=1.10
 tested-with:
     GHC == 8.0.2,
     GHC == 8.2.2,
@@ -76,7 +76,7 @@ library
         Data.Time.Format.ISO8601,
         Data.Time
     other-modules:
-        Data.Format
+        Data.Format,
         Data.Time.Calendar.Types,
         Data.Time.Calendar.Private,
         Data.Time.Calendar.Days,
@@ -96,7 +96,7 @@ library
         Data.Time.Clock.Internal.UTCDiff,
         Data.Time.LocalTime.Internal.TimeZone,
         Data.Time.LocalTime.Internal.TimeOfDay,
-        Data.Time.LocalTime.Internal.CalendarDiffTime
+        Data.Time.LocalTime.Internal.CalendarDiffTime,
         Data.Time.LocalTime.Internal.LocalTime,
         Data.Time.LocalTime.Internal.ZonedTime,
         Data.Time.Format.Parse,
@@ -110,6 +110,8 @@ library
         install-includes:
             HsTime.h
     else
+        autogen-includes:
+            HsTimeConfig.h
         install-includes:
             HsTime.h
             HsTimeConfig.h

(With newer cabal-version parser is also pickier about commas, if you choose to use ones in exposed-modules, they should be between every element).

[AshleyYakeley]

It seems like the cabal format has become stricter in recent versions, and the time package no longer complies, is that correct?

[Bodigrim]

@phadej thanks! Would not autogen-includes: HsTimeConfig.hs be enough? Otherwise, shall one add autogen-includes: HsTime.hs to if os(windows) branch as well?

[phadej]

@Bodigrim yes, just HsTimeConfig.h.

EDIT: Update the above diff.

[phadej]

@AshleyYakeley Not exactly. cabal v2-build is stricter, and isolates build environments better. Format is the same, and newer versions have added fields to let explicitly declare previously implicit things. (Cabal just used to look for files, and try to install them).

Also as part of v2-build cabal registers the package in (project) local package database. That is the step where things fail, as HsTimeConfig.h is not found.

In fact, without the patch above cabal sdist fails to make a source distribution to begin with!

% cabal sdist
cabal: can't find include file HsTimeConfig.h

autogen-includes: (and other autogen- fields) tell cabal sdist to not look for these, but trust that they will be generated by configure (or some other means). Also makes install look for files in folders where autogenerated files are put.

[phadej]

For the record: I haven't made the patch earlier (e.g. when the Cabal issue was closed), because Hackage didn't support uploading cabal-version: 3.0 packages then. I think it's now possible, (yet, I haven't tried).

[Bodigrim]

I think it's now possible, (yet, I haven't tried).

It is possible. Even base-4.14.1.0 requires cabal-version: 3.0, and there are many more examples.
https://hackage-search.serokell.io/?q=cabal-version%3A%5Cs*3.0

[AshleyYakeley]

OK, I can fix this (or you can make a PR). I'll make a 1.11.1.2 release.

[phadej]

I don't think release is necessary. It affects only local builds, so you can wait until the release is needed otherwise.

I'll make a PR right away

[AshleyYakeley]

OK, fix checked in. I'm going to make a 1.11.1.2 version anyway. Thanks everyone.

[Bodigrim]

time is also rebuilt, when building a project involving a patched boot library (bytestring in my case). A release would be much appreciated.

[phadej]

time is also rebuilt, when building a project involving a patched boot library (bytestring in my case).

How so? That seems like a bug in cabal-install logic. Are you using cabal-install-3.4? (Which changed how source-repository-package are handled?)

[Bodigrim]

Why would it be a bug? time depends on bytestring via Win32.

[phadej]

Ah. Windows. Who uses that for development :)

(FWIW, release won't help with rebuilding then. You'll rebuild newer time still).

[Bodigrim]

Well, I don't mind rebuilding as long as it succeeds.

[AshleyYakeley]

There are problems building with older GHC versions. This might take awhile to figure out.

[phadej]

There are problems building with older GHC versions. This might take awhile to figure out.

What you mean by that?

[AshleyYakeley]

It means don't expect a release today...

[phadej]

I don't understand. CI is green, also locally I have

      9.0.0.20201227  8.10.3  8.8.4  8.6.5  8.4.4  8.2.2  8.0.2  7.10.3  7.8.4  7.6.3  7.4.2  7.2.2  7.0.4
time  OK              OK      OK     OK     OK     OK     OK     NO-IP   NO-IP  NO-IP  NO-IP  NO-IP  NO-IP

(This is on Linux)

[phadej]

@AshleyYakeley I suggest that you consider @Bodigrim's offer about CI, https://github.com/haskell/bytestring/runs/1619829279?check_suite_focus=true looks quite nice (and checks Windows too, e.g.).

(@Bodigrim https://github.com/haskell/actions/tree/main/setup should AFAIU support all GHCs supported by hvr-ppa, it tries various sources incl ppa, ghcup and preinstalled ones, so you won't need to split into old/new GHC).

[AshleyYakeley]

The problem is it's no longer possible to install time-1.11.1.2 with an older GHC using stack. This is because version 3 of the Cabal library needs to be installed first, but that itself depends on the time library.

Previously, this wasn't a problem, because the stock Cabal that came with GHC (back to 8.0.2) was always capable of reading the time.cabal file. This isn't the case in the current master branch.

[AshleyYakeley]

Might have to revert the PR. Reopening.

[AshleyYakeley]

That's true, and that's certainly a motivation to upgrade time.cabal to version 3 format. Currently cabal sdist doesn't work unless you run configure first.

Note that there is no v1-sdist, which leaves cabal users without an alternative at all.

Who runs cabal sdist besides me when I'm preparing a new release?

[Bodigrim]

Currently cabal sdist doesn't work unless you run configure first.

Yes, and then sdist distributes your locally generated HsTimeConfig.h. For instance, when I unpack time-1.11.1.1.tar.gz, I receive HsTimeConfig.h with outdated entries like #define PACKAGE_VERSION "1.10", because that was probably the last time you ran configure locally. Luckily, cabal seems to insist on re-running configure again, regenerating headers.

Who runs cabal sdist besides me when I'm preparing a new release?

Most probably nobody :) But sdist could be used more often than that, it is often a part of CI workflow, like git clone; cabal sdist; cabal get; cabal test to ensure that sdist contains everything for reproducible build. So someone maintaining a private fork of time could theoretically be affected.

[AshleyYakeley]

Luckily, cabal seems to insist on re-running configure again, regenerating headers.

Sadly that doesn't help, I forgot to run autoconf... (#148)

[AshleyYakeley]

OK, I've asked for community feedback here on the possibility of dropping support for old GHC versions.

[Bodigrim]

I do not quite understand why inability to use Stack with GHC <= 8.6 necessitates dropping support of old GHCs altogether. Is it because Cabal has not been able to build tests before #147?

[AshleyYakeley]

If people are using old GHCs with new time, I'd prefer to support stack. In that case, I will revert time.cabal to the old version.

[Bodigrim]

I'll accept your decision as a maintainer on this matter, so just a few data points:

• People using old GHCs with new time can always resort to time-compat, this is its very raison d'etre.

• It is a question for @phadej, but since the issue deals with the isolation of build environments, I doubt Cabal will provide a workaround. We can postpone cabal-version: 3.0 a bit later, but not indefinitely.

[AshleyYakeley]

This is still a regression in cabal-install, is it not? Older versions of cabal-install work with released versions of time, while cabal-install version 3 does not.

[phadej]

All cabal-install work with released versions of time. This issue doesn't manifest itself when package is installed as a dependency from Hackage.

The issue is that time cannot be used as a local package, i.e. part of a project.

In other words, it's impossible for cabal users to contribute to the development of time.

[AshleyYakeley]

I see. I use stack to build the time library locally, partly because it's easier to manage multiple GHC versions.

Generally, I would like to move time.cabal to version 3 if I can. If there is no-one who is using older GHCs with newer time, then I can drop support for older GHC versions. This would not only solve this problem, but also let me remove some of the CPP conditional code.

[AshleyYakeley]

@Bodigrim, why are you building your own time library from the repo? Maybe there's an alternative solution for what you're trying to do?

[Bodigrim]

I run bytestring test suite. Building local bytestring from the repo triggers rebuilds of all boot libraries, depending on bytestring, and on Windows it includes time. So even while I grab time directly from Hackage and not from the repo, it is still a "local build" and fails as described.

There are certain workarounds available indeed. E. g., I can recommend bytestring contributors to resort to cabal v1-build when using Windows, and setup CI accordingly. But this approach does not scale well: if I wish to test any other package or application on Windows against a yet unreleased bytestring (as a part of release preparation, for example), I'm in trouble because of v1-build limitations (probably one can conjure a sandbox or something, but...)

[AshleyYakeley]

For as long as time supports GHC 8.6 and earlier, time.cabal must use the old format. As far as I can tell, the time.cabal file in releases is correct within that format.

It seems to me that version 3 of cabal-install is not properly backwards-compatible with the old .cabal format, and that this is a defect in cabal-install.

[Bodigrim]

It's true that cabal new-build is not fully backward-compatible with cabal v1-build. Otherwise there would have been no reason for separate commands and no transition period to adapt to new workflows. This is by design.

I use stack to build the time library locally, partly because it's easier to manage multiple GHC versions.

I used to think the same, but it appeared that ghcup is absolutely up to the task.

[AshleyYakeley]

Has cabal new-build always failed with the time library? Or is this something new with version 3 of cabal-install?

[Bodigrim]

This is not something new with Cabal 3.X series, because new-build fails in Cabal 2.4 as well.

[AshleyYakeley]

Not sure I can help you then. At some point I will drop support for GHC 8.6, and at that point I will migrate time.cabal to version 3.

[Bodigrim]

Given recent changes in master, could we possibly get back to this?

[phadej]

If support for older GHCs is dropped please remember to update lower bounds.

[AshleyYakeley]

Done.