Does lorri work with nix flakes?

[matthuszagh]

Based on the instructions here and here, direnv is able to support flakes with a use_flake() function. However, when I tried this in a project with a flake.nix and flake.lock lorri complained about the lack of a shell.nix file.

Does lorri support nix flakes? If so, how can it be used? If not, is support planned?

[thufschmitt]

I don't think lorri supports flakes atm, but you can use flake-compat to create a shell.nix that lorri can use, with

(import (fetchTarball https://github.com/edolstra/flake-compat/archive/master.tar.gz) {
  src = ./.;
}).shellNix.default

[Profpatsch]

Flakes are not a stable feature at the moment, they are not even mentioned in the nix manual.

We could start adding support for them, but afaik the format changes every few weeks at the moment, I don’t see the big selling point of interacting with experimental features.

[thufschmitt]

I think it would be a good time to start thinking about flakes for lorri − even without implementing it now − as the fact that it and the new cli are both experimental atm means that it's possible to upstream some lorri-specific logic into Nix itself (which will be much more painful once it gets stable, if only because lorri will have to be backwards compatible with the Nix-with-flakes-but-without-the-changes-that-make-it-easier-to-implement).

I'm not super-familiar with the internals of lorri, but my understanding of how lorri works is that it does essentially three things on the Nix side:

1. It builds an overriden version of the shell.nix where the builder is replaced by something that will just dump the environment
2. It adds a gc root for this overriden derivation (so that its inputs stays alive)
3. During the evaluation of this derivation, it hijacks a few builtins ((ab)using scopedImport) to make them trace their own execution (to know which files to watch)

On the Nix side,

1. Nix now has nix print-dev-env which seems to do the same thing
2. nix print-dev-env doesn't have a way to add a gc root, but it could be easily added (it's just a matter of plumbing as the code already has everything for that, the gc root is just discarded at the very last moment)
3. I didn't follow the latest developments on that front, but I think this hijacking is now redundant with some native logging in Nix (if not I guess it's time to make a PR to add it)

If the above is true, it means that supporting flakes in lorri would be reasonably simple, provided we make a couple of changes to Nix before it reaches a stable version. At a very high level, Lorri would essentially look like:

LORRI_ROOT=~/.cache/lorri/roots/foobar

# Dump the environment for the derivation, adding a gc root at $LORRI_ROOT and saving the log into nix-exec.log
LORRI_ENV=$(nix print-dev-env --add-root $LORRI_ROOT --log-format json 2> nix-exec.log)
# Get the list of the files read by Nix during the evaluation from the log
READ_FILES=$(cat nix-exec.log | jq 'some magic to extract all the read files from the log')
# Rebuild everything if one of these change
inotifywait --event modify $READ_FILES
exec "$0" "$@"

@Profpatsch do I over-simplify stuff or does that look like a reasonable plan?

[nyarly]

As I read it, this proposal misses the mark. Lorri does a real nix-instantiate of the shell.nix; the builder is left in place, but (granted) scopedImport hacks in instrumentation for file loads. print-dev-env does seem to behave as if the builder were stubbed out; but that's not what we want.

That said, the core of the suggestion seems good. If the nix core team (notably Eelco) were convinced that Lorri was valuable enough to consider in their roadmap, it would be helpful. Concretely, getting in-built support for logging requirements (instead of interpositioning instrumentation) would be great, and being an acknowledged stakeholder in the design would both be great.

I'm personally increasingly interested in Flakes, and I'd love to be able to use them with Lorri. Much as the caching provided there is valuable, the operations being cached aren't free and backgrounding them is its own benefit.

[thufschmitt]

Lorri does a real nix-instantiate of the shell.nix; the builder is left in place, [...] print-dev-env does seem to behave as if the builder were stubbed out; but that's not what we want.

Does it? My understanding is that it instantiates keep-env-hack (import shell.nix), which seems to do essentially the same thing as nix prent-dev-env does (well keep-env-hack is a tad bit more complex, but I expect that both could be unified if needs be).

That said, the core of the suggestion seems good. If the nix core team (notably Eelco) were convinced that Lorri was valuable enough to consider in their roadmap, it would be helpful. Concretely, getting in-built support for logging requirements (instead of interpositioning instrumentation) would be great, and being an acknowledged stakeholder in the design would both be great.

I can't say about the broad roadmap, but unless there's a hidden tradeoff that I can't see, the logging bit at least is definitely something that can be added.

[Profpatsch]

@regnat do you have a good and/or complete description of the semantics of nix dev-env?

As you mentioned, the lorri keep-env-hack also prints all dependencies that changed. In order for lorri to not be as hackish, there should be a way to query nix for this information.

[thufschmitt]

@regnat do you have a good and/or complete description of the semantics of nix dev-env?

Not much more than Nix's help message itself (or skimming through the source code)

$ nix print-dev-env --help
[...]
Summary: print shell code that can be sourced by bash to reproduce the build environment of a derivation. 
[...]
Examples:

  To apply the build environment of GNU hello to the current shell:
  $ . <(nix print-dev-env nixpkgs#hello)
[...]

As you mentioned, the lorri keep-env-hack also prints all dependencies that changed. In order for lorri to not be as hackish, there should be a way to query nix for this information.

Do you mean the logging of import, readFile and friends? Or is there something more?

Something that keep-env-hack does though that (afaik) nix print-dev-env doesn't is to add gc roots. But I also think that it's something worth adding to Nix regardless of lorri

[FlorianFranzen]

If you run a flake enabled nix by default, the following shell.nix alongside your flake.nix can also be used:

{ system ? builtins.currentSystem }:

(builtins.getFlake (toString ./.)).devShell.${system}

[deifactor]

If @FlorianFranzen's approach works, would it be possible to just do that as a temporary measure while a more principled approach to flakes is set up? Like, if there's a flake.nix but no shell.nix, treat it as if shell.nix contained a compatibility shim.

[gilescope]

@FlorianFranzen approach seems to work for me so I've raised a PR (to the right repo) so that it's suggested if a shell.nix isn't found but a flake.nix is.

[GaetanLepage]

Hello ! Any updates on the support of nix flakes ?
Thanks again to the developers of this amazing little tool !