Flakes: Don't Require a system Suffix

[theduke]

I'm not sure if an issue is the right place for this discussion.
If not, please feel free to close and direct me to a better venue.

Problem

The current structure of flakes, with derivations in a hierarchy that includes the system, is pretty annoying to use.

• It basically mandates using a helper library like flake-utils

• It adds complexity to an already complex system, and makes writing flakes more awkward.
  It also introduces relatively confusing error messages.

• Most packages will be the same across systems, with potentially only small modifications.

Solutions

I think optimising developer experience for the common case is important.

I suggest:

• Just use a plain, system-less hierarchy as the default
• Prefer derivations with a system suffix , if present.

That retains the flexibility of the current approach, but doesn't introduce the additional burden when not required.

[edolstra]

Not including the system type is incompatible with hermetic evaluation, since it means that evaluating an attribute like packages.foo will produce a different result on different systems. It also means that you don't have a name to unambiguously refer to a flake output, so you wouldn't be able to use a i686-linux package as a dependency of a x86_64-linux package.

[theduke]

Would it be possible to do some kind of auto-expansion?

Eg: if devShell is not a map, but a derivation, auto-expand it into a map that covers all common systems.

[roberth]

Not including the system type is incompatible with hermetic evaluation, since it means that evaluating an attribute like packages.foo will produce a different result on different systems.

What if we had a primop that behaves as follows but memoizes its result:

functionPath: argPaths: import functionPath (mapAttrs import argPaths)

This doesn't correspond neatly to flakes, but is more powerful. Wouldn't we be able to create a functionPath that wires up a flake?
Another place where it could be applied is a new flake attribute, for example: perSystem = system: { defaultPackage = (nixpkgs.perSystem system).packages.hello; ..... } ie allowing packages to be proper functions of system, as they are, rather than a finite map. After all, many flakes have no business listing their supported systems, because they have no reason not to (try to) support all of them.
This solves one part of the problem. The other is ease of use. Instead of wiring up perSystem ourselves, Flakes could wire it for us by automatically invoking it such that self.perSystem = system: outputs.perSystem system (mapAttrs (input: input.perSystem system) inputs). Now the package wiring becomes as simple as

outputs = inputs@{...}: {
  perSystem = system: { nixpkgs, ... }: { defaultPackage = nixpkgs.packages.hello; };
};

No helper lib required, no unnecessary system enumeration and still able to use packages compiled on/for a different system, by invoking inputs.nixpkgs.perSystem "i686-linux" for example.

By giving the memoization to users, we can also experiment with cross compilation in flakes, which doesn't seem to have been a consideration in the flake design yet.

(Apologies if I sound repetitive; couldn't find my earlier comment)

[xaverdh]

Can't flakes have functions as output? Then you can already fall back to just providing the function (System -> ...) itself?

[roberth]

@xaverdh, yes they can. Such a fallback doesn't scale though. The ecosystem is already referencing foo.${system} system everywhere, so all those flakes just break. That's why Nix needs to provide the new standard.

(Apologies if I sound repetitive; couldn't find my earlier comment)

Haven't found the comment, but this is basically a duplicate of #3843.

[stale]

I marked this as stale due to inactivity. → More info

[roberth]

I'd make this problem top priority.

Flakes should natively support both system-agnostic attributes like templates and host-system-dependent attributes like packages in two outputs "sections". These are two mutually recursive values: one attrset for selecting systems and one function to attrset for generalizing systems.

This is part of the domain, and by neglecting it, we are confusing users and creating a sprawl of helper functions.
Flakes should take the lead and standardize this.

[RossComputerGuy]

I recently discussed this with @roberth after implementing #12426. Another concept of implementation is with adding builtins.currentPlatform which defines an attr set. Would look something like this:

{
  system = "aarch64-linux";
  cc = "gcc";
  libc = "musl";
  pageSize = {
    min = 4096;
    max = 16384;
  };
}

This would then get passed to nixpkgs's lib.systems.elaborate which with NixOS/nixpkgs#365057 implemented (plus other future work), it would tell nixpkgs what the local platform is like. This would make using specialized configurations better (ie pkgsLLVM or pkgsMusl systems). However, this would add a new problem with Flakes where it wouldn't work well into an attr set key. So I've thought of a new function Flakes could implement called lib.resolveFlakeAttrSystem. It takes in two arguments, system and kind. System being builtins.currentPlatform and kind being what kind of attribute it wants to resolve (ie devShells, packages, etc.). It would return null for nothing to resolve or a list of strings being the attribute set path to look in.

I'm thinking it might be good to gather 2 or 3 concepts and discuss them among the Nix team, come together with an RFC and work things through there. From the RFC would come the full implementation.

[roberth]

Many packages are agnostic to system, toolchain, ABI, etc, so it would be beneficial to support flake authors to pass such information along to Nixpkgs and other flakes with little effort.
Here's a related discussion about using functions for this purpose

• Extend the flake / cli interface with a function for system-specific attributes #7709

By using a function, it is easy to pass all this configuration to another input's configuring function.