llvmPackages_latest: move to aliases.nix #229852

alyssais · 2023-05-04T08:54:29Z

Description of changes

Because llvmPackages_latest is used in Nixpkgs, by quite a few packages, it's difficult to keep it up to date, because updating it requires some level of confidence that every package that uses it is going to keep working after the update. The result of this is that llvmPackages_latest is not updated, and so we end up in the situation that "latest" is two versions older than the latest version we actually provide. This is confusing and unexpected.

"But won't this end up fragmenting our LLVM versions, if every package previously using _latest is separately pinned to LLVM 14?", I hear you ask. No. That fragmentation is already happening, even with an llvmPackages_latest, because packages that actually require the latest version of LLVM (15/16), have already been decoupled from llvmPackages_latest since it hasn't been upgraded. So like it or not, we can't escape packages depending on specific recent LLVMs. The only real fix is to get better at keeping the default LLVM up to date (which I'm reasonably confident we're getting into a better position to be feasibly better able to do).

So, unless we want to double down on providing a confusingly named "llvmPackages_latest" attribute that refers to some arbitrary LLVM version that's probably not the latest one (or even the latest one available in Nixpkgs), we only have two options here: either we don't provide such an attribute at all, or we don't use it in Nixpkgs so we don't become scared to bump it as soon as we have a new LLVM available.

Things done

RaitoBezarius

Agreed with the rationale.

rrbutani

I wholeheartedly agree with the spirit of this change.

I think it makes sense to have packages that are fairly closely coupled with LLVM update their LLVM version at their own cadence and I think it makes sense that deviating from llvmPackages is a good indicator that a package is somewhat closely coupled to LLVM. For packages in the latter category I think it's actually better that they pin an LLVM version than use _latest; bumping LLVM in these packages can be involved and should be something maintainers of these packages can do on their own timeline (I think #213202 and #225197 are a good example of this).

I think the only potential downside to this change is: the packages that aren't super sensitive to their LLVM version that have been able to benefit passively from us bumping llvmPackages_latest will no longer be able to do so if they're ever made to use something other than llvmPackages. For example: if an infrequently maintained package is made to use a newer-than-llvmPackages version but then isn't bumped for many years.

But as @alyssais notes, the way to mitigate this is to bump llvmPackages more aggressively (to reduce the need to pin to newer LLVM versions) and also to perhaps periodically try to prune uses of older LLVM package sets (as @RaitoBezarius and others have been trying to do recently).

I have a couple of nits (packages that aren't sensitive to LLVM version for which I think we should use llvmPackages rather than pinning, lest we forget to update the pinned version — assuming we're okay with the version of LLVM used by these packages/aliases going backwards for a little bit) but nothing that I feel strongly about.

rrbutani · 2023-05-04T16:59:57Z

pkgs/top-level/all-packages.nix

@@ -14540,7 +14540,7 @@ with pkgs;
  clang_16 = llvmPackages_16.clang;

  clang-tools = callPackage ../development/tools/clang-tools {
-    llvmPackages = llvmPackages_latest;
+    llvmPackages = llvmPackages_14;


Can we make this llvmPackages instead of pinning to llvmPackages_14? This package (just a wrapper) isn't sensitive to changes in LLVM and this seems like exactly the kind of thing we'd forget to bump.

Suggested change

llvmPackages = llvmPackages_14;

I think people probably want a recent clang for clangd, clang-tidy, etc. I'd rather bump this to 16.

rrbutani · 2023-05-04T17:02:10Z

pkgs/top-level/all-packages.nix

+    llvmPackages = llvmPackages_14;
+    inherit (llvmPackages_14) clang;


Ditto with this package; just another wrapper.

Suggested change

llvmPackages = llvmPackages_14;

inherit (llvmPackages_14) clang;

rrbutani · 2023-05-04T17:09:36Z

pkgs/top-level/all-packages.nix

@@ -15547,7 +15547,7 @@ with pkgs;
  lld_15 = llvmPackages_15.lld;
  lld_16 = llvmPackages_16.lld;

-  lldb = llvmPackages_latest.lldb;
+  lldb = lldb_14;


lldb was an outlier in that unlike the other tool aliases (clang, lld, etc) it pointed to llvmPackages_latest instead of llvmPackages.

I believe the motivation (#99365) was to have a newer lldb version by default; if we're planning to bump llvmPackages more aggressively I think we can safely set this to:

Suggested change

lldb = lldb_14;

lldb = llvmPackages.lldb;

Though this would mean that for now lldb in nixpkgs will go backwards from v14 to v11/v12 (depending on platform); I'm not sure if that's acceptable.

rrbutani · 2023-05-04T17:17:41Z

pkgs/top-level/all-packages.nix

@@ -15589,7 +15587,7 @@ with pkgs;
      else if platform.isAndroid then 12
      else if platform.isLinux then 11
      else if platform.isWasm then 12
-      else latest_version;
+      else 14;


This seems like (imo) the one place where picking the latest version might actually be what we want? (assuming that newest LLVM is our best bet at providing support for other/untested platforms)

On the other hand there probably aren't many users relying on this fallback anyways (even if we forget to bump this it's probably okay) and I'd rather see us gain the ability to specify the LLVM version as part of the system config anyways. So I think this is fine.

alyssais · 2023-05-04T19:31:28Z

I have a couple of nits (packages that aren't sensitive to LLVM version for which I think we should use llvmPackages rather than pinning, lest we forget to update the pinned version — assuming we're okay with the version of LLVM used by these packages/aliases going backwards for a little bit) but nothing that I feel strongly about.

These are all valid suggestions, but the point of this work is that we don't need to make decisions about multiple packages at once in a single PR, so in that spirit I'm not going to be making any of those changes here.

alyssais · 2023-05-04T19:33:32Z

I think the only potential downside to this change is: the packages that aren't super sensitive to their LLVM version that have been able to benefit passively from us bumping llvmPackages_latest will no longer be able to do so if they're ever made to use something other than llvmPackages. For example: if an infrequently maintained package is made to use a newer-than-llvmPackages version but then isn't bumped for many years.

This is true in theory, but not in practice, because in practice we have not been bumping llvmPackages_latest at all. It came close to not even getting bumped to 14.

sternenseemann · 2023-05-05T20:36:44Z

pkgs/os-specific/linux/keyutils/default.nix

@@ -18,7 +18,7 @@ stdenv.mkDerivation rec {
    ./conf-symlink.patch
    # This patch solves a duplicate symbol error when building with a clang stdenv
    # Before removing this patch, please ensure the package still builds by running eg.
-    # nix-build -E 'with import ./. {}; pkgs.keyutils.override { stdenv = pkgs.llvmPackages_latest.stdenv; }'
+    # nix-build -E 'with import ./. {}; pkgs.keyutils.override { stdenv = pkgs.llvmPackages_14.stdenv; }'


Default seems to make more sense here?

Agreed. I've changed it to pkgs.clangStdenv.

Because llvmPackages_latest is used in Nixpkgs, by quite a few packages, it's difficult to keep it up to date, because updating it requires some level of confidence that every package that uses it is going to keep working after the update. The result of this is that llvmPackages_latest is not updated, and so we end up in the situation that "latest" is two versions older than the latest version we actually provide. This is confusing and unexpected. "But won't this end up fragmenting our LLVM versions, if every package previously using _latest is separately pinned to LLVM 14?", I hear you ask. No. That fragmentation is already happening, even with an llvmPackages_latest, because packages that actually require the _latest_ version of LLVM (15/16), have already been decoupled from llvmPackages_latest since it hasn't been upgraded. So like it or not, we can't escape packages depending on specific recent LLVMs. The only real fix is to get better at keeping the default LLVM up to date (which I'm reasonably confident we're getting into a better position to be feasibly better able to do). So, unless we want to double down on providing a confusingly named "llvmPackages_latest" attribute that refers to some arbitrary LLVM version that's probably not the latest one (or even the latest one available in Nixpkgs), we only have two options here: either we don't provide such an attribute at all, or we don't use it in Nixpkgs so we don't become scared to bump it as soon as we have a new LLVM available.

alyssais · 2023-05-08T12:20:44Z

This should be merged soon after OfBorg is happy, because since I opened it two more uses of llvmPackages_latest appeared that I've had to fix.

alyssais added the 6.topic: policy discussion label May 4, 2023

alyssais requested review from lovek323, RaitoBezarius, K900, dtzWill, Ericson2314, sternenseemann, primeos and rrbutani May 4, 2023 08:54

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` and removed 6.topic: policy discussion labels May 4, 2023

ofborg bot added 8.has: clean-up 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels May 4, 2023

wegank approved these changes May 4, 2023

View reviewed changes

RaitoBezarius approved these changes May 4, 2023

View reviewed changes

rrbutani added the 6.topic: policy discussion label May 4, 2023

rrbutani approved these changes May 4, 2023

View reviewed changes

rrbutani mentioned this pull request May 4, 2023

llvmPackages_latest: 14 -> 15 #213202

Closed

13 tasks

Ericson2314 approved these changes May 5, 2023

View reviewed changes

sternenseemann approved these changes May 5, 2023

View reviewed changes

mweinelt added the 12.approvals: 3+ This PR was reviewed and approved by three or more reputable people label May 6, 2023

alyssais force-pushed the llvmPackages_latest-alias branch from 76754c2 to 3b6d31f Compare May 8, 2023 12:14

github-actions bot removed the 6.topic: policy discussion label May 8, 2023

alyssais added 2 commits May 8, 2023 12:20

llvmPackages_latest: 14.0.6 -> 16.0.1

3b88760

alyssais force-pushed the llvmPackages_latest-alias branch from 3b6d31f to 3b88760 Compare May 8, 2023 12:20

alyssais merged commit da41baa into NixOS:master May 8, 2023

alyssais deleted the llvmPackages_latest-alias branch May 8, 2023 13:05

rrbutani added the 6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related label May 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

llvmPackages_latest: move to aliases.nix #229852

llvmPackages_latest: move to aliases.nix #229852

alyssais commented May 4, 2023

RaitoBezarius left a comment

rrbutani left a comment •

edited

Loading

rrbutani May 4, 2023

alyssais May 4, 2023

rrbutani May 4, 2023

rrbutani May 4, 2023

rrbutani May 4, 2023

alyssais commented May 4, 2023

alyssais commented May 4, 2023

sternenseemann May 5, 2023

alyssais May 8, 2023

alyssais commented May 8, 2023

		llvmPackages = llvmPackages_14;
		inherit (llvmPackages_14) clang;

llvmPackages_latest: move to aliases.nix #229852

llvmPackages_latest: move to aliases.nix #229852

Conversation

alyssais commented May 4, 2023

Description of changes

Things done

RaitoBezarius left a comment

Choose a reason for hiding this comment

rrbutani left a comment • edited Loading

Choose a reason for hiding this comment

rrbutani May 4, 2023

Choose a reason for hiding this comment

alyssais May 4, 2023

Choose a reason for hiding this comment

rrbutani May 4, 2023

Choose a reason for hiding this comment

rrbutani May 4, 2023

Choose a reason for hiding this comment

rrbutani May 4, 2023

Choose a reason for hiding this comment

alyssais commented May 4, 2023

alyssais commented May 4, 2023

sternenseemann May 5, 2023

Choose a reason for hiding this comment

alyssais May 8, 2023

Choose a reason for hiding this comment

alyssais commented May 8, 2023

rrbutani left a comment •

edited

Loading