Support GHC 9.0.1 (at least in some packages)

[sjakobi]

Blocked on:

• Build failures:
  • Cannot build basement with GHC 9.0: unsafeCoerce# no longer exported by GHC.Prim in base 4.15+ haskell-foundation/foundation#548 (basement <--- {cryptonite,http-client-tls,memory} <- dhall)
  • Support for GHC 9.0.1 DanBurton/lens-family-th#18 (lens-family-th <- hnix <- dhall-nix)
  • lens-family <- hnix <- dhall-nix (Contacted via email)
  • Doesn't build with GHC-9.0 mrkkrp/modern-uri#34 (modern-uri <- mmark <- dhall-docs)
  • Build failure with GHC 9.0.1 commercialhaskell/path#174 (path <- dhall-docs)
  • Please, support GHC 9.0.x haskell-nix/hnix#858 (hnix <- dhall-nix)
  • GHC 9.0 and cabal-install 3.4 sol/doctest#275
  • hnix-store-core fails to build with GHC 9.0.1 haskell-nix/hnix-store#142 (hnix-store-core <- hnix <- dhall-nix)
  • Doesn't compile on GHC 9.0.1 due to unsafeCoerce# vincenthz/hs-memory#84 (memory <- {cryptonite,dhall,http-client-tls})
• Bounds:
  • New releases? well-typed/cborg#260
  • Bounds + CI for GHC 9.0.1 chrisdone/lucid#104
  • Support for GHC 9.0 haskell-hvr/cryptohash-sha512#7 (via hnix-store-core)
  • Support for GHC 9.0 haskell-hvr/cryptohash-sha256#12 (via hnix-store-core)
  • Support for GHC 9.0.1 haskell-hvr/cryptohash-sha1#10 (via hnix-store-core)
  • Support for GHC 9.0.1 haskell-hvr/cryptohash-md5#5 (via hnix-store-core)
  • Support for GHC 9.0.1 haskell-hvr/HsYAML-aeson#9
  • Support for GHC 9.0 haskell-hvr/HsYAML#54
  • Support for GHC 9.0.1 haskell-hvr/hslogger#61 (via haskell-lsp)
  • Support for GHC 9.0.1 mitchellwrosen/tasty-hspec#23

The incantation for circumventing the bounds issues is:

cabal build -O0 all --enable-tests --enable-benchmarks --allow-newer=haskell-lsp:base,haskell-lsp-types:base,hnix:hashable,hnix:template-haskell,hnix:semialign-indexed,hnix:semialign,cryptohash-sha512:base,cryptohash-sha256:base,cryptohash-sha1:base,cryptohash-md5:base,hslogger:base

[Anton-Latukha]

haskell-hvr/cryptohash-sha is passed from hnix-store?

hnix would have a big release, there is big migration ongoing. Estimate release in ~2 months, maybe a ~1 month maybe faster. Seems like trying to chip at 0.9 worth this release.

I was doing hnix-store migration to cryptonite, but so far I postponed work in favor of HNix refactors in the main computational path.

BTW, hnix-store-core is under removal from Stackage Nightly, and from new LTSes.

With current resources I give estimate of ~3 month until theHNix migration to GHC 9.0

BTW, because of big changes in HNix, besides of giving a good ChangeLog, I probably would double in helping migrating dhall-nix to the new HNix releases once they come out, especially definitely the next one. Because dhall-nix is the main alive well-used project in HNix deps - that strategy gives HNix team the freedom to do huge code improvements in it.

[sjakobi]

haskell-hvr/cryptohash-sha is passed from hnix-store?

Indeed. We depend on the cryptohash-* packages only via hnix-store-core.

I was doing hnix-store migration to cryptonite

Yeah, cryptonite doesn't seem to be very actively maintained either, but it doesn't look as dire as HVR's cryptohash-* packages. dhall uses cryptonite directly too.

BTW, because of big changes in HNix, besides of giving a good ChangeLog, I probably would double in helping migrating dhall-nix to the new HNix releases once they come out, especially definitely the next one. Because dhall-nix is the main alive well-used project in HNix deps - that strategy gives HNix team the freedom to do huge code improvements in it.

Thanks! If you could give us a hand with the adjustments in dhall-nix (and possibly in dhall-nixpkgs too), I'd really appreciate it! :)

[Gabriella439]

@antislava: Yeah, don't worry about make breaking changes to the hnix API. We can easily accommodate those changes

[sjakobi]

At this stage, haskell-foundation/foundation#548 appears to be the only true blocker for GHC 9.0.1 compatibility in the core dhall package. Since the basement maintainer has been so unresponsive, I've been wondering how we could remove that dependency.

The direct dependencies that use basement are: cryptonite, http-client-tls, memory, where cryptonite and memory AFAIK are only used to produce and parse SHA256 digests.

• http-client-tls could be replaced with http-client-openssl
• cryptonite could be replaced with SHA
• the base16-parsing functionality of memory could be replaced with base16 or base16-bytestring

My main concern about this switch is that SHA might be slower than crytonite.

In any case, I don't plan on executing this switch until GHC 9.0.2 has been released – GHC 9.0.1 has a serious correctness bug.

[Gabriella439]

@sjakobi: I think it would be fine to switch to those packages, including SHA. From a quick scan of the SHA source code it seems like they pay careful attention to performance in the implementation, so I'm not too concerned about a performance regression.

[sjakobi]

Now that Vincent Hanquez has uploaded compatible versions of basement, memory and cryptonite, dhall, dhall-bash, dhall-json, dhall-yaml and dhall-openapi build with GHC 9.0.1 and their tasty test suites pass.

dhall-docs builds, but its golden test suite produces different output due to a change in the Hashable Text instance in hashable-1.3.1.0. I've opened https://github.com/chrisdone/lucid/issues/107 for this and added a bound hashable < 1.3.1 to this test suite.

dhall-nix and dhall-nixpkgs will need a compatible hnix release.

dhall-lsp-server will need to migrate from the old haskell-lsp package to the new lsp package (see http://hackage.haskell.org/package/lsp-1.0.0.1/changelog).

I think we should merge and release the progress that we have so far.

[sjakobi]

Hydra failed due to

Dhall Tests
  QuickCheck
    everything well-typed should normalize:                                                                                    FAIL (31.38s)
      *** Failed! (after 82619 tests and 1 shrink):
      Exception:
        Error: Compiler bug

        An ill-typed expression was encountered during normalization.
        Explanation: This error message means that there is a bug in the Dhall compiler.
        You didn't do anything wrong, but if you would like to see this problem fixed
        then you should report the bug at:

        https://github.com/dhall-lang/dhall-haskell/issues

        CallStack (from HasCallStack):
          error, called at src/Dhall/Eval.hs:1201:13 in dhall-1.38.1-BNLcpta7e8d3IQfIPD3r7g:Dhall.Eval
      App TextReplace (TextLit (Chunks [] ""))
      Use --quickcheck-replay=126743 to reproduce.

Looks like a bug, but probably not due to this PR.

[Anton-Latukha]

GHC9.0: (hnix-store-{core,remote} release were made ~ month ago & migrated away from cryptohash-* to cryptonite) & hnix release just was made supporting all that, that means the majority of the list is covered.

& guys went over and made a set of revisions for HVR packages, hslogger also got new revision allowing GHC 9.0.

[sjakobi]

The direct dependencies that use basement are: cryptonite, http-client-tls, memory, where cryptonite and memory AFAIK are only used to produce and parse SHA256 digests.

• http-client-tls could be replaced with http-client-openssl
• cryptonite could be replaced with SHA
• the base16-parsing functionality of memory could be replaced with base16 or base16-bytestring

The direct dependencies on cryptonite and memory were removed in #2335. The transitive dependency on basement via http-client-tls remains TODO.

[Anton-Latukha]

Well, "just great", hnix-store migrated away from cryptohash-sha512 to cryptonite, dhall-haskell migrated to cryptohash-sha256.

Is there some upstream consensus? When phadej needs to also maintain cryptohash-sha512 & cryptonite stays single-handedly vincent - I do not feel safe with neither of those "solutions". I see no people were given maintainers privileges, seems like those projects learned nothing.

---

Btw, I have grown tired on : continuously work on HNix & pursuing Nix upstream currently - (when no one seems interested) & went to do more meaningful work, so ping me - or, - some people do have access.