Wrong current path with custom build-type

[hanjoosten]

My project has a custom build-type. After setup.hs is built, the resulting setup.exe is executed. In the past, it got executed in the project directory (the directory where the .cabal file resides). That is also the root directory of git.
One of the tasks of the custom setup is to fetch the commit id of git. However, this cannot be found now. Is this a bug, or is this a change on purpose? If the latter, is this documented somewhere?

• Here is the build log at appveyor
• Here are the sources at the point of this build

NB: A build of the same code with stack works fine

[hanjoosten]

This issue could be solved if there is a way to get the project's root directory programmatically. Then that could be passed as a parameter to the process getting the git info. However, I could not find this in the documentation. Any pointers would help a lot.

[hvr]

@zw3rk this was the intentional effect of #4874 wasn't it?

[phadej]

I suggest looking into cabal-doctest on how to generate autogen-modules, if that's what you are doing: https://github.com/phadej/cabal-doctest/blob/master/src/Distribution/Extra/Doctest.hs

I'm surprised if cabal-install changed directory. cabal-doctest build hook is run in the package directory. I use makeAbsolute and never had problems with it.

[phadej]

Ok. now I looked more closely: v2-install sdists the package before installing; therefore .git repository is not there. I don't know a way around it.

[gbaz]

hrm could one add .git/FETCH_HEAD or .git/ORIG_HEAD or whatever as a datafile to the cabal file directly? :-)

[phadej]

@gbaz it's not so easy, you'll need to chase pointers;

~/mess/Ampersand development % cat .git/HEAD                 
ref: refs/heads/development
~/mess/Ampersand development % cat .git/refs/heads/development
580b0f2ede9f19f33472290516f576c99833c1b6

And we have the limitation https://cabal.readthedocs.io/en/latest/developing-packages.html#pkg-field-data-files

If a wildcard is used, it must be used with an extension, so data-files: data/* is not allowed.

[gbaz]

right, which is why I suggested FETCH_HEAD or ORIG_HEAD though I confess I don't fully understand the structure involved :-)

[hanjoosten]

This is probably related with issue #5001

[hvr]

@hanjoosten not necessarily; based on @phadej's observation this is an intentional effect of cabal v2-install which goes via a deterministic sdist in order to be able to compute a stable hash for the nix-style store. And we also had to sacrifice support for the sdist-hooks as those would have made the implementation a lot more complex.

In general, inferring the Git hash & tags during package builds is a terribly fragile thing to do and personally I've given up 10 years ago fighting with cabal (doing such Git-hackery was the very first thing I ever did with custom Setup.hs scripts and it quickly became very complex as I had to rewrite the .cabal file and Setup.hs when doing a sdist to get rid of Git invocation which must never be part of the sdist artifact for obvious reasons) to do this inside cabal as it kept breaking and causing more trouble than it's worth especially when trying to make a build-system as precise as possible as it ends up thrashing your build everytime and slows things down. Also, there's also cabal features you can't access anymore or are forced into a degraded mode as soon as you require a custom Setup.hs. Consequently, I've moved over to deal things like these by using external "boot"-scripts which let me keep build-type: Simple packages which cause a lot less trouble. Problems which I consider a symptom showing that the Git operation is really a concern that IMO doesn't belong into Cabal but is an external concern that ought to be handled externally to Cabal's layer.

[hanjoosten]

My usecase is all about tracability. When I build foo.exe, I want to be able to know the exact codebase that was used to build it. So if the executable is called like foo.exe --version, I want the output to be a reference to that codebase. Using git, a reference to the commit id seems very reasonable to use. I could live with some other mechanism to maintain this tracability. For instance some kind of hash that is generated at the creation time of sdist, and that is available during setup. There should then be some way to use that hash to find the original commit in the version system used.

[hvr]

@hanjoosten Yours is the very use-case I had 10 years ago and still have to this day! I need to be able in principle to exactly reproduce the state of the source tree and build-plan was in of a given executable artifact. And this is also the problem I'm currently solving via use of external scripts that produce the metadata, instead of having cabal drive the process. I.e. I have a script to make a deliverable artifact, which takes care of making sure the source tree is not dirty and that everything is in fact properly identifiable by the commit hash (which also covers the cabal freeze file, so the git commit hash also implicitly identifies the version of dependencies that went into the build).

This is even easier to do combined with a CI/CD buildbot ideally sealed off from the internet which produces the artifacts, which ensures that each build starts from a clean state. All I'm trying to say here is that you don't need to do this inside the cabal layer; you can do this quite easily externally before cabal takes over.

[hanjoosten]

@hvr Hmm. I see what you mean. This could indeed be done by doing some preprocessing before calling cabal. I didn't want to go that way because you made a point last week about not using preprocessing.

[hvr]

@hanjoosten that's not necessarily a contradiction :-)

In fact, the arguments I brought up there apply here as well: if you need the ability to use the remote git repo deps (which is typically more relevant for libraries than for executables; and the executables are the ones you'd typically want to embed git hashes into), as you'd want for public OSS projects, then pre-processing like what I suggest needs to make sure the project still builds in a reasonable possibly gracefully degraded way (i.e. w/o Git metadata). In fact, it's desirable to have this also for local development (and while I didn't mention this, this is actually something I sometimes do as well for inhouse projects), as you certainly don't want a git hash change to invalidate/thrash your local build everytime it changes.

[tom-audm]

@hanjoosten when #4746 is fixed you can use the gitrev package [0] to get this --version behavior without a custom Setup.hs

(see #5866)

[0] https://hackage.haskell.org/package/gitrev-1.3.1/docs/Development-GitRev.html

[hanjoosten]

@tom-audm I doubt that, because you still need the .git directory available. As you can read in the replies of @hvr, that is no longer the case. Nor at build time, let alone at runtime.