From 274eb695e4d94d97f0293d46df6ea8a797558ec7 Mon Sep 17 00:00:00 2001 From: Robert Hensing Date: Thu, 4 Jul 2024 17:44:14 +0200 Subject: [PATCH 1/3] nixpkgs: Allow passing {host,build}Platform directly instead of *[sS]ystem This new interface has already been applied successfully in NixOS. `nixos-generate-hardware` has been generating the "default" hostPlatform in `hardware-configuration.nix` for over a year now [without much trouble], and with the benefit of not having to specify `system` (or similar) in `nixosSystem` anymore. Furthermore, the `hostPlatform` option is always defined and reliably produces the host platform for users who don't use the legacy options. This is in contrast to `nixpkgs.crossSystem`, which is usually not even defined, and `nixpkgs.localSystem` which is usually the wrong platform to refer to in a config file. Ideally we'd clean up the `nixpkgs.{system,localSystem,crossSystem}` options to reduce complexity and confusion. But the interface in Nixpkgs is still based on the old terminology and behavior. By introducing these parameters also in Nixpkgs, the users' experience with NixOS carries over to Nixpkgs as well. Further simplifications in the code base are now possible, specifically - stage.nix and related files still work with {local,cross}System, and have logic like ${if stdenv.hostPlatform == stdenv.buildPlatform then "localSystem" else "crossSystem"} = <...> ... which is really just hostPlatform = <...> This can now be simplified by refactoring this code to work with {host,build}Platform variables instead. - NixOS can forward its platform options directly to its Nixpkgs call. This pays off when the `*[sS]ystem` options are removed. [without much trouble]: https://github.com/NixOS/nixpkgs/pull/237218 --- doc/using-nixpkgs.md | 1 + doc/using/as-a-function.chapter.md | 151 +++++++++++++++++++++++++ pkgs/test/top-level/default.nix | 174 +++++++++++++++++++++++++++++ pkgs/top-level/default.nix | 74 ++++++++++-- pkgs/top-level/impure.nix | 38 ++++++- 5 files changed, 424 insertions(+), 14 deletions(-) create mode 100644 doc/using/as-a-function.chapter.md diff --git a/doc/using-nixpkgs.md b/doc/using-nixpkgs.md index f850b2e83c28d..de98334bfbcaa 100644 --- a/doc/using-nixpkgs.md +++ b/doc/using-nixpkgs.md @@ -2,6 +2,7 @@ ```{=include=} chapters using/platform-support.chapter.md +using/as-a-function.chapter.md using/configuration.chapter.md using/overlays.chapter.md using/overrides.chapter.md diff --git a/doc/using/as-a-function.chapter.md b/doc/using/as-a-function.chapter.md new file mode 100644 index 0000000000000..b28d08009d020 --- /dev/null +++ b/doc/using/as-a-function.chapter.md @@ -0,0 +1,151 @@ +# Nixpkgs as a Function {#sec-nixpkgs-function} + +Depending on how you use Nixpkgs, you may interact with it as a [Nix function]. +The Nixpkgs function returns an attribute set containing mostly packages, but also other things, such as more package sets, and functions. +This return value is often referred to with the identifier `pkgs`, and it has a marker attribute, `_type = "pkgs";`. + +The Nixpkgs function can be retrieved in multiple ways. + +In a setup with [Nix channels], you may look up the Nixpkgs files with [``][Nix lookup path]; for example + +```console +$ nix repl -f '' +Added 21938 variables. +nix-repl> +``` + + +[`nix repl`] [`-f`][`nix repl -f`] automatically imports the Nixpkgs function and invokes it for you. + +Most Nix commands accept [`--arg`] and [`--argstr`] options to pass arguments to the Nixpkgs function. + +In an expression, you can use the `import { }` syntax to import the Nixpkgs function, and immmediately [apply] it. + +```nix +let + pkgs = import { + config = { allowUnfree = true; }; + }; +in +``` + +Note that these examples so far have relied on the implicit, [impure] [`builtins.currentSystem`] variable to configure the package set to be able to run on your system. + +If you wish not to accidentally rely on the implicit [`builtins.currentSystem`] variable, and you'd like to avoid configuration in `~/.config/nixpkgs`, you may invoke `` instead. + +To make sure you can reproduce your evaluations, you may use [pinning] or locking, such as provided by [`npins`] or [Flakes]. + +## Arguments {#sec-nixpkgs-arguments} + +For historical reasons, the platform and cross compilation parameters can be specified in multiple ways. + +If you run Nix in pure mode, or if you work with expressions for multiple host platforms, you are recommended to use `hostPlatform` for native compilation (locally or on a remote machine), and both `hostPlatform` and `buildPlatform` for cross compilation. + +### `hostPlatform` and `buildPlatform` {#sec-nixpkgs-arguments-platforms} + +- `hostPlatform` is the platform for which the package set is built. This means that the binaries in the outputs are compatible with `hostPlatform`. + + You may specify this as a cpu-os string, such as `"x86_64-linux"`, or `aarch64-darwin`, or you can pass a more details platform object produced with `lib.systems`. + +- `buildPlatform` (default: `hostPlatform`) is the platform on which the derivations will run. The derivations produced by Nixpkgs will have a [`system` attribute] that matches `buildPlatform`, so that builds are sent to a machine that is capable of running the `builder` command, which is also configured to be compatible with `buildPlatform`. + + Values are specified in the same formats as `hostPlatform`. + +::: {.example #ex-nixpkgs-pure-native} + +# Natively compiled packages + +This shows how to explicitly request packages that are built for a specific platform, not relying on [`builtins.currentSystem`]. +The packages will be built natively. + +If you request to build this on a host that is not compatible, you need a remote builder. + +```nix +let + pkgs = imports { + hostPlatform = "x86_64-linux"; + }; +in pkgs.hello +``` + +::: + +::: {.example #ex-nixpkgs-pure-cross} + +# Cross compiled packages + +This expression will produce packages that are built on `x86_64-linux` machines, but whose outputs can run on `riscv64-linux`. + +The expression is pure, so it will produce the same outcome even if run on e.g. an `aarch64-darwin` machine. For that to work, the macOS machine needs the packages to be available in a cache or it needs a remote builder that can build `x86_64-linux` derivations. + +If you were to use [`builtins.currentSystem`] for `buildPlatform`, you would be able to build the packages directly on any machine, but the resulting store paths will be different, depending on the current system. +Especially if you are in a team that doesn't run a single CPU architecture and operating system or you use Nix on machines of more than one platform, you might notice that equivalent deployments would have different store paths, resulting in unnecessary rebuilds, more store path copying, and unnecessary updates to equivalent configurations with different `buildPlatform`s. +For these reason, it is recommended to use pure configurations where the platform parameters are set to fixed values for each deployment target. + +```nix +let + pkgs = imports { + hostPlatform = "riscv64-linux"; + buildPlatform = "x86_64-linux"; + }; +in pkgs.hello +``` + +::: + +### Legacy arguments `system`, `localSystem` and `crossSystem` {#sec-nixpkgs-arguments-systems-legacy} + +These parameters existed before [`hostPlatform` and `buildPlatform`](#sec-nixpkgs-arguments-platforms) were introduced. +They are still supported, but are not recommended for new code. + +The main difference is that these parameters default to [`builtins.currentSystem`], which is not recommended for pure code. + +Furthermore, their design is optimized for impure command line use. + +Unlike `hostPlatform`, `crossSystem` is not always set, so it does not make for a nice abstraction of compilation. + + +For example, [`nixos-generate-config`] wasn't initially able to set the platform in `hardware-configuration.nix` without making assumptions about cross versus native compilation, resulting in a need for you to manually specify `system` when Flakes were introduced. + +### `config` {#sec-nixpkgs-arguments-config} + +Example: + +```nix +import { config = { allowUnfree = true; }; } +``` + +See [Configuration](#sec-config-options-reference) for details. + +### `overlays` {#sec-nixpkgs-arguments-overlays} + +Overlays are expressions that modify the package set. + +See [Overlays](#chap-overlays) for more information. + +### `crossOverlays` {#sec-nixpkgs-arguments-crossOverlays} + + +Cross overlays apply only to the final package set in cross compilation. +This means that [`buildPackages`] and [`nativeBuildInputs`] are unaffected by these overlays and this increases the number of build dependencies that can be retrieved from the cache, or can be reused from a previous build. + +[Nix function]: https://nix.dev/manual/nix/latest/language/constructs.html#functions +[Nix channels]: https://nix.dev/manual/nix/latest/command-ref/nix-channel.html +[Nix lookup path]: https://nix.dev/manual/nix/latest/language/constructs/lookup-path.html +[`builtins.currentSystem`]: https://nix.dev/manual/nix/latest/language/builtin-constants.html#builtins-currentSystem +[`system` attribute]: https://nix.dev/manual/nix/latest/language/derivations#attr-system +[`--arg`]: https://nix.dev/manual/nix/latest/command-ref/opt-common.html?highlight=--arg#opt-arg +[`--argstr`]: https://nix.dev/manual/nix/latest/command-ref/opt-common.html?highlight=--argstr#opt-argstr +[apply]: https://nix.dev/manual/nix/latest/language/operators +[impure]: https://nix.dev/manual/nix/latest/command-ref/conf-file.html?highlight=pure-eval#conf-pure-eval +[`npins`]: https://nix.dev/guides/recipes/dependency-management.html +[Flakes]: https://nix.dev/concepts/flakes.html#flakes +[pinning]: https://nix.dev/reference/pinning-nixpkgs.html + +[`nixos-generate-config`]: https://nixos.org/manual/nixos/stable/#sec-installation + +[`buildPackages`]: #ssec-cross-dependency-implementation + +[`nativeBuildInputs`]: #ssec-stdenv-dependencies-propagated +[`nix repl`]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-repl.html +[`nix repl -f`]: https://nix.dev/manual/nix/latest/command-ref/new-cli/nix3-repl?highlight=--file#opt-file diff --git a/pkgs/test/top-level/default.nix b/pkgs/test/top-level/default.nix index fdb9fe09a88b8..4bbe0ce552412 100644 --- a/pkgs/test/top-level/default.nix +++ b/pkgs/test/top-level/default.nix @@ -1,3 +1,4 @@ +# nix-build -A tests.top-level { lib, pkgs, ... }: let nixpkgsFun = import ../../top-level; @@ -44,4 +45,177 @@ lib.recurseIntoAttrs { assert lib.all (p: p.buildPlatform == p.hostPlatform) pkgsLocal; assert lib.all (p: p.buildPlatform != p.hostPlatform) pkgsCross; pkgs.emptyFile; + + invocations-pure = + let + inherit (lib.systems) elaborate; + assertEqualPkgPlatforms = a: b: + # NOTE: equals should become obsolete, but currently isn't when comparing + # between two distinct Nixpkgs invocations + # `equals` should not be used in most places because it is a bit slow. + assert lib.systems.equals a.stdenv.hostPlatform b.stdenv.hostPlatform; + assert lib.systems.equals a.stdenv.buildPlatform b.stdenv.buildPlatform; + assert lib.systems.equals a.stdenv.targetPlatform b.stdenv.targetPlatform; + true; + + native-s = import ../../.. { system = "i686-linux"; }; + native-ls = import ../../.. { localSystem.system = "i686-linux"; }; + native-le = import ../../.. { localSystem = lib.systems.elaborate "i686-linux"; }; + native-h = import ../../.. { hostPlatform = "i686-linux"; }; + native-he = import ../../.. { hostPlatform = lib.systems.elaborate "i686-linux"; }; + + cross-1-s-cs = import ../../.. { system = "i686-linux"; crossSystem = "armv7l-linux"; }; + cross-1-s-css = import ../../.. { system = "i686-linux"; crossSystem.system = "armv7l-linux"; }; + cross-1-s-cse = import ../../.. { system = "i686-linux"; crossSystem = elaborate "armv7l-linux"; }; + cross-1-ls-cs = import ../../.. { localSystem.system = "i686-linux"; crossSystem = "armv7l-linux"; }; + cross-1-ls-css = import ../../.. { localSystem.system = "i686-linux"; crossSystem.system = "armv7l-linux"; }; + cross-1-ls-cse = import ../../.. { localSystem.system = "i686-linux"; crossSystem = elaborate "armv7l-linux"; }; + cross-1-le-cs = import ../../.. { localSystem = elaborate "i686-linux"; crossSystem = "armv7l-linux"; }; + cross-1-le-css = import ../../.. { localSystem = elaborate "i686-linux"; crossSystem.system = "armv7l-linux"; }; + cross-1-le-cse = import ../../.. { localSystem = elaborate "i686-linux"; crossSystem = elaborate "armv7l-linux"; }; + cross-1-b-h = import ../../.. { buildPlatform = "i686-linux"; hostPlatform = "armv7l-linux"; }; + cross-1-b-hs = import ../../.. { buildPlatform = "i686-linux"; hostPlatform.system = "armv7l-linux"; }; + cross-1-b-he = import ../../.. { buildPlatform = "i686-linux"; hostPlatform = elaborate "armv7l-linux"; }; + cross-1-bs-h = import ../../.. { buildPlatform.system = "i686-linux"; hostPlatform = "armv7l-linux"; }; + cross-1-bs-hs = import ../../.. { buildPlatform.system = "i686-linux"; hostPlatform.system = "armv7l-linux"; }; + cross-1-bs-he = import ../../.. { buildPlatform.system = "i686-linux"; hostPlatform = elaborate "armv7l-linux"; }; + cross-1-be-h = import ../../.. { buildPlatform = elaborate "i686-linux"; hostPlatform = "armv7l-linux"; }; + cross-1-be-hs = import ../../.. { buildPlatform = elaborate "i686-linux"; hostPlatform.system = "armv7l-linux"; }; + cross-1-be-he = import ../../.. { buildPlatform = elaborate "i686-linux"; hostPlatform = elaborate "armv7l-linux"; }; + + in + + # NATIVE + # ------ + + assert lib.systems.equals native-s.stdenv.hostPlatform (elaborate "i686-linux"); + assert lib.systems.equals native-s.stdenv.buildPlatform (elaborate "i686-linux"); + + # sample reflexivity (sanity check) + assert assertEqualPkgPlatforms native-s native-s; + # check the rest (assume transitivity to avoid n^2 checks) + assert assertEqualPkgPlatforms native-s native-ls; + assert assertEqualPkgPlatforms native-s native-le; + assert assertEqualPkgPlatforms native-s native-h; + assert assertEqualPkgPlatforms native-s native-he; + + # These pairs must be identical + assert native-s.stdenv.hostPlatform == native-s.stdenv.buildPlatform; + assert native-ls.stdenv.hostPlatform == native-ls.stdenv.buildPlatform; + assert native-le.stdenv.hostPlatform == native-le.stdenv.buildPlatform; + assert native-h.stdenv.hostPlatform == native-h.stdenv.buildPlatform; + assert native-he.stdenv.hostPlatform == native-he.stdenv.buildPlatform; + + + # CROSS + # ----- + + assert lib.systems.equals cross-1-s-cs.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-s-cs.stdenv.buildPlatform (elaborate "i686-linux"); + + # sample reflexivity (sanity check) + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-s-cs; + # check the rest (assume transitivity to avoid n^2 checks) + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-s-css; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-s-cse; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-ls-cs; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-ls-css; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-ls-cse; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-le-cs; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-le-css; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-le-cse; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-b-h; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-b-hs; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-b-he; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-bs-h; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-bs-hs; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-bs-he; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-be-h; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-be-hs; + assert assertEqualPkgPlatforms cross-1-s-cs cross-1-be-he; + + assert lib.systems.equals cross-1-s-cs.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-s-css.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-s-cse.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-ls-cs.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-ls-css.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-ls-cse.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-le-cs.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-le-css.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-le-cse.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-b-h.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-b-hs.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-b-he.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-bs-h.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-bs-hs.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-bs-he.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-be-h.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-be-hs.stdenv.hostPlatform (elaborate "armv7l-linux"); + assert lib.systems.equals cross-1-be-he.stdenv.hostPlatform (elaborate "armv7l-linux"); + + assert lib.systems.equals cross-1-s-cs.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-s-css.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-s-cse.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-ls-cs.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-ls-css.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-ls-cse.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-le-cs.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-le-css.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-le-cse.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-b-h.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-b-hs.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-b-he.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-bs-h.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-bs-hs.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-bs-he.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-be-h.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-be-hs.stdenv.buildPlatform (elaborate "i686-linux"); + assert lib.systems.equals cross-1-be-he.stdenv.buildPlatform (elaborate "i686-linux"); + + # builtins.trace '' + # Tests passed, but note that the impure use cases are not included here. + + # Impurely specified native compilation, + # currentSystem as build -> currentSystem as host: + + # nix-build -A hello + + # Impurely specified cross compilation for specified host, + # currentSystem as build -> crossSystem as host: + + # nix-build --argstr crossSystem armv7l-linux -A hello + + # '' + + + + pkgs.emptyFile; + + invocations-impure = + if !builtins?currentSystem + then + lib.trace + "Skipping invocations-impure test, because we don't have currentSystem here. Don't forget to also run the top-level tests in impure mode!" + pkgs.emptyFile + else + let + # As a mere expression, we don't get to pick currentSystem, so we have to + # pick a different one dynamically. + otherSystem = if builtins.currentSystem == "x86_64-linux" then "aarch64-linux" else "x86_64-linux"; + otherSystemElaborated = lib.systems.elaborate otherSystem; + currentSystemElaborated = lib.systems.elaborate builtins.currentSystem; + + impureNative = import ../../.. { }; + impureCross = import ../../.. { crossSystem = otherSystem; }; + in + # Obviously + assert ! lib.systems.equals currentSystemElaborated otherSystemElaborated; + + assert lib.systems.equals impureNative.stdenv.hostPlatform currentSystemElaborated; + assert lib.systems.equals impureNative.stdenv.buildPlatform currentSystemElaborated; + + assert lib.systems.equals impureCross.stdenv.hostPlatform otherSystemElaborated; + assert lib.systems.equals impureCross.stdenv.buildPlatform currentSystemElaborated; + + pkgs.emptyFile; } diff --git a/pkgs/top-level/default.nix b/pkgs/top-level/default.nix index 5c224802d5bf3..ceb0ac603a404 100644 --- a/pkgs/top-level/default.nix +++ b/pkgs/top-level/default.nix @@ -16,15 +16,33 @@ evaluation is taking place, and the configuration from environment variables or dot-files. */ -{ # The system packages will be built on. See the manual for the +let + # This is not pkgs/top-level/impure.nix or default.nix, so we'll need at least something. + noArgumentError = + abort '' + You must specify the hostPlatform argument to nixpkgs, to specify the platform on which the packages will run. + Example: + nixpkgs { hostPlatform = "x86_64-linux"; } + ''; +in +{ + # Where the packages will run + # TODO: document in the manual + hostPlatform ? noArgumentError, + + # Where the derivations are built. Default: hostPlatform. + # TODO: document in the manual + buildPlatform ? hostPlatform, + + # (legacy) The system packages will be built on. See the manual for the # subtle division of labor between these two `*System`s and the three # `*Platform`s. - localSystem + localSystem ? noArgumentError, -, # The system packages will ultimately be run on. - crossSystem ? localSystem + # (legacy) The system packages will ultimately be run on. + crossSystem ? localSystem, -, # Allow a configuration attribute set to be passed in as an argument. + # Allow a configuration attribute set to be passed in as an argument. config ? {} , # List of overlays layers used to extend Nixpkgs. @@ -49,16 +67,42 @@ let # Rename the function arguments in let lib = import ../../lib; - inherit (lib) throwIfNot; + inherit (lib) intersectAttrs throwIfNot throwIf; + + legacySystemArgsUsed = intersectAttrs { system = null; localSystem = null; crossSystem = null; } args; + + intentToCrossCompile = + # is it legacy cross? + (args?localSystem && !lib.systems.equals crossSystem localSystem) + || # is it new cross? + (args?buildPlatform && args?hostPlatform && !lib.systems.equals crossSystem localSystem); checked = throwIfNot (lib.isList overlays) "The overlays argument to nixpkgs must be a list." lib.foldr (x: throwIfNot (lib.isFunction x) "All overlays passed to nixpkgs must be functions.") (r: r) overlays throwIfNot (lib.isList crossOverlays) "The crossOverlays argument to nixpkgs must be a list." lib.foldr (x: throwIfNot (lib.isFunction x) "All crossOverlays passed to nixpkgs must be functions.") (r: r) crossOverlays + throwIf (args?buildPlatform && !args?hostPlatform) '' + nixpkgs: You have only specified buildPlatform, but not hostPlatform. + - For cross compilation: specify both + - For native compilation: just specify hostPlatform; buildPlatform defaults to it + '' + throwIf (args?hostPlatform && (legacySystemArgsUsed != {})) '' + nixpkgs: You have specified hostPlatform, but also one of the system, localSystem, or crossSystem. + Mixing legacy system arguments and hostPlatform is not supported. + ${if intentToCrossCompile then '' + For cross compilation, please specify the nixpkgs arguments + - hostPlatform: the platform on which the packages will run + - buildPlatform: the platform on which the derivations are built + '' else ""} + If you are not cross-compiling, please specify only hostPlatform. + '' ; - localSystem = lib.systems.elaborate args.localSystem; + localSystem = + if args?hostPlatform + then lib.systems.elaborate buildPlatform + else lib.systems.elaborate args.localSystem; # Condition preserves sharing which in turn affects equality. # @@ -73,10 +117,18 @@ in let # Both systems are semantically equivalent as the same vendor and ABI are # inferred from the system double in `localSystem`. crossSystem = - let system = lib.systems.elaborate crossSystem0; in - if crossSystem0 == null || lib.systems.equals system localSystem - then localSystem - else system; + if args?hostPlatform + then + if lib.systems.equals (lib.systems.elaborate buildPlatform) (lib.systems.elaborate hostPlatform) + then + # If equal, make them identical, so `==` works on this pair of platforms. + localSystem + else (lib.systems.elaborate hostPlatform) + else + let system = lib.systems.elaborate crossSystem0; in + if crossSystem0 == null || lib.systems.equals system localSystem + then localSystem + else system; # Allow both: # { /* the config */ } and diff --git a/pkgs/top-level/impure.nix b/pkgs/top-level/impure.nix index 4d847e280f4b9..c2a272a7bfda0 100644 --- a/pkgs/top-level/impure.nix +++ b/pkgs/top-level/impure.nix @@ -11,7 +11,17 @@ let in -{ # We put legacy `system` into `localSystem`, if `localSystem` was not passed. +{ + # Where the packages will run + hostPlatform ? + # unreachable + abort "argument `hostPlatform` is optional in top-level/impure.nix, but has no default, because it and related arguments are processed in top-level/default.nix instead", + # Where the derivations are built + buildPlatform ? + # unreachable + abort "argument `buildPlatform` is optional in top-level/impure.nix, but has no default, because it and related arguments are processed in top-level/default.nix instead", + + # We put legacy `system` into `localSystem`, if `localSystem` was not passed. # If neither is passed, assume we are building packages on the current # (build, in GNU Autotools parlance) platform. localSystem ? { system = args.system or builtins.currentSystem; } @@ -84,6 +94,28 @@ in assert args ? localSystem -> !(args ? system); assert args ? system -> !(args ? localSystem); +# # Similarly for more recently soft-deprecated parameters, in which case we +# # don't want to mix and match: +# # Either use the pre 24.11 legacy attrs, or {host,build}Platform. +# assert (args ? localSystem || args ? system || args ? crossSystem) != (args ? hostPlatform || args ? buildPlatform); +let + # If legacy args are passed, we need to perform the localSystem default logic here. + # If they are not passed, + hasLegacyArgs = args ? system || args ? localSystem || args ? crossSystem; + impureArgs = !args ? system && !args ? localSystem && !args ? crossSystem && !args ? hostPlatform && !args ? buildPlatform; +in + import ./. (builtins.removeAttrs args [ "system" ] // { - inherit config overlays localSystem; -}) + inherit config overlays; + } + // (if hasLegacyArgs || impureArgs + then { + # default.nix is a bit simpler than impure.nix (here), as we only perform the + # defaulting behavior here... + inherit localSystem; + } else { + # ... however, for the new {host,build}Platform, we + # don't want to trigger the checks in default.nix or weaken those checks. + } + ) +) From eaa3ff4461fbd545c0698d235f37697a34055d0c Mon Sep 17 00:00:00 2001 From: Robert Hensing Date: Mon, 30 Sep 2024 09:44:49 +0200 Subject: [PATCH 2/3] doc/as-a-function: Edit --- doc/using/as-a-function.chapter.md | 7 +++++-- pkgs/top-level/default.nix | 7 ++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/doc/using/as-a-function.chapter.md b/doc/using/as-a-function.chapter.md index b28d08009d020..97a709edd761b 100644 --- a/doc/using/as-a-function.chapter.md +++ b/doc/using/as-a-function.chapter.md @@ -98,12 +98,15 @@ in pkgs.hello These parameters existed before [`hostPlatform` and `buildPlatform`](#sec-nixpkgs-arguments-platforms) were introduced. They are still supported, but are not recommended for new code. -The main difference is that these parameters default to [`builtins.currentSystem`], which is not recommended for pure code. +The main difference is that these parameters default to [`builtins.currentSystem`], which does not work for pure code. Furthermore, their design is optimized for impure command line use. -Unlike `hostPlatform`, `crossSystem` is not always set, so it does not make for a nice abstraction of compilation. +The term "local" is misleading, because it need not match the platform where the Nix command is run. +The term "cross" is also misleading because its value may equal `localSystem`. + +Finally `crossSystem` may have to be unset as an input, complicating some code that calls the Nixpkgs function. For example, [`nixos-generate-config`] wasn't initially able to set the platform in `hardware-configuration.nix` without making assumptions about cross versus native compilation, resulting in a need for you to manually specify `system` when Flakes were introduced. diff --git a/pkgs/top-level/default.nix b/pkgs/top-level/default.nix index ceb0ac603a404..417cae9bab118 100644 --- a/pkgs/top-level/default.nix +++ b/pkgs/top-level/default.nix @@ -17,7 +17,8 @@ or dot-files. */ let - # This is not pkgs/top-level/impure.nix or default.nix, so we'll need at least something. + # This file has pure behavior, unlike pkgs/top-level/impure.nix or default.nix, + # so we can't infer everything from the expression language environment. noArgumentError = abort '' You must specify the hostPlatform argument to nixpkgs, to specify the platform on which the packages will run. @@ -27,11 +28,11 @@ let in { # Where the packages will run - # TODO: document in the manual + # See doc/using/as-a-function.chapter.md hostPlatform ? noArgumentError, # Where the derivations are built. Default: hostPlatform. - # TODO: document in the manual + # See doc/using/as-a-function.chapter.md buildPlatform ? hostPlatform, # (legacy) The system packages will be built on. See the manual for the From 4fd048a763a7fd02f20f0c59918e24fab3e19215 Mon Sep 17 00:00:00 2001 From: Robert Hensing Date: Mon, 30 Sep 2024 09:49:32 +0200 Subject: [PATCH 3/3] doc/as-a-function: Command line us of system and crossSystem are ok --- doc/using/as-a-function.chapter.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/using/as-a-function.chapter.md b/doc/using/as-a-function.chapter.md index 97a709edd761b..c02693f311227 100644 --- a/doc/using/as-a-function.chapter.md +++ b/doc/using/as-a-function.chapter.md @@ -95,6 +95,8 @@ in pkgs.hello ### Legacy arguments `system`, `localSystem` and `crossSystem` {#sec-nixpkgs-arguments-systems-legacy} +`crossSystem` and `system` (or `localSystem`) are ok to use on the command line, but are not recommended for pure Nix code (such as Nix flakes), or Nix code that manages the invocation of the Nixpkgs function. + These parameters existed before [`hostPlatform` and `buildPlatform`](#sec-nixpkgs-arguments-platforms) were introduced. They are still supported, but are not recommended for new code.