Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/options: Warn about options without type #127066

Closed
wants to merge 2 commits into from

Conversation

dasJ
Copy link
Member

@dasJ dasJ commented Jun 16, 2021

This is a follow-up to #76184

Closes #76184
Closes #141730
Closes #141760

Motivation for this change

Should push people to actually add types to all options.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@dasJ dasJ requested review from edolstra, infinisil and nbp as code owners June 16, 2021 10:29
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Jun 16, 2021
@bbjubjub2494
Copy link
Member

bbjubjub2494 commented Jun 16, 2021

The relevant documentation should be updated since it says that the option type may be omitted.

https://nixos.org/manual/nixos/unstable/index.html#sec-option-declarations

EDIT: https://github.com/lourkeur/nixpkgs/tree/doc/option-type

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation This PR adds or changes documentation labels Jun 16, 2021
@ofborg ofborg bot added 10.rebuild-linux: 1-10 and removed 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Jun 16, 2021
@infinisil
Copy link
Member

To prevent options without a type for all options, not just ones rendered in documentation, you should add the warning here instead:

then opt // { type = f (opt.type or types.unspecified); }

@dasJ dasJ force-pushed the feat/warn-no-type branch from 5e97ba3 to a742176 Compare September 8, 2021 19:12
@dasJ
Copy link
Member Author

dasJ commented Sep 8, 2021

@infinisil I tried that but there are actually a lot of internal options that have no type. Going for all options in the manual is a good start in my opinion without having to touch a lot of options.
Adding that second warning results in:

$ nix-build --check nixos/tests/simple.nix 2>&1 | grep -c 'has no type'
375

@infinisil
Copy link
Member

It turns out that this PR didn't work at all like this, I could never make it produce any warnings even with an option that obviously doesn't have a type.

I force pushed a fix based on my previous suggested, but making sure to only throw a warning for options that aren't internal and are visible.

This now gives a couple warnings:

$ nix-instantiate nixos --arg configuration nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix -A system
trace: warning: Option `isoImage.volumeID' defined in `/home/infinisil/src/nixpkgs/nixos/modules/installer/cd-dvd/iso-image.nix' has no type.
trace: warning: Option `isoImage.edition' defined in `/home/infinisil/src/nixpkgs/nixos/modules/installer/cd-dvd/iso-image.nix' has no type.
trace: warning: Option `dysnomia.components' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/misc/dysnomia.nix' has no type.
trace: warning: Option `dysnomia.containers' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/misc/dysnomia.nix' has no type.
trace: warning: Option `dysnomia.extraContainerPaths' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/misc/dysnomia.nix' has no type.
trace: warning: Option `dysnomia.extraContainerProperties' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/misc/dysnomia.nix' has no type.
trace: warning: Option `dysnomia.extraModulePaths' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/misc/dysnomia.nix' has no type.
trace: warning: Option `dysnomia.properties' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/misc/dysnomia.nix' has no type.
trace: warning: Option `hardware.pcmcia.config' defined in `/home/infinisil/src/nixpkgs/nixos/modules/hardware/pcmcia.nix' has no type.
trace: warning: Option `security.pam.loginLimits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `services.hoogle.haskellPackages' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/development/hoogle.nix' has no type.
trace: warning: Option `services.unbound.stateDir' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/networking/unbound.nix' has no type.
trace: warning: Option `services.vsftpd.userlist' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/networking/vsftpd.nix' has no type.
trace: warning: Option `services.xserver.displayManager.session' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/x11/display-managers/default.nix' has no type.
trace: warning: Option `services.xserver.windowManager.xmonad.haskellPackages' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/x11/window-managers/xmonad.nix' has no type.
trace: warning: Option `security.pam.services.<name>.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `services.nsd.zones.<name>.children' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/networking/nsd.nix' has no type.
trace: warning: Option `specialisation.<name>.configuration' defined in `/home/infinisil/src/nixpkgs/nixos/modules/system/activation/top-level.nix' has no type.
trace: warning: Option `security.pam.services.chfn.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.loginLimits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.chpasswd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.chsh.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupadd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupdel.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupmems.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupmod.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.i3lock.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.i3lock-color.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.login.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.passwd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.polkit-1.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.runuser.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.runuser-l.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.sshd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.su.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.sudo.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.systemd-user.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.useradd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.userdel.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.usermod.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.vlock.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.xlock.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.xscreensaver.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `services.xserver.displayManager.session' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/x11/display-managers/default.nix' has no type.
trace: warning: Option `installer.cloneConfig' defined in `/home/infinisil/src/nixpkgs/nixos/modules/profiles/clone-config.nix' has no type.
trace: warning: Option `installer.cloneConfigIncludes' defined in `/home/infinisil/src/nixpkgs/nixos/modules/profiles/clone-config.nix' has no type.
trace: warning: Option `installer.cloneConfigExtra' defined in `/home/infinisil/src/nixpkgs/nixos/modules/profiles/clone-config.nix' has no type.

And some more depending on which modules are included:

$ nix-instantiate nixos/tests/simple.nix
trace: warning: Option `virtualisation.vlans' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.writableStore' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.writableStoreUseTmpfs' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.useBootLoader' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `security.pam.services.chfn.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.loginLimits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.chpasswd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.chsh.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupadd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupdel.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupmems.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.groupmod.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.i3lock.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.i3lock-color.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.login.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.passwd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.polkit-1.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.runuser.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.runuser-l.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.su.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.sudo.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.systemd-user.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.useradd.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.userdel.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.usermod.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.vlock.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.xlock.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `security.pam.services.xscreensaver.limits' defined in `/home/infinisil/src/nixpkgs/nixos/modules/security/pam.nix' has no type.
trace: warning: Option `services.xserver.displayManager.session' defined in `/home/infinisil/src/nixpkgs/nixos/modules/services/x11/display-managers/default.nix' has no type.
trace: warning: Option `virtualisation.diskImage' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.diskSize' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.memorySize' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.useEFIBoot' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.graphics' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
trace: warning: Option `virtualisation.pathsInNixDB' defined in `/home/infinisil/src/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix' has no type.
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/gwd6dk720gw0spb1xvs1h4h5ab7l2mqp-vm-test-run-simple.drv

@infinisil
Copy link
Member

Oh wait, just realized that the force push didn't work, looks like I can't push to this branch.

Pushed it to my own branch: https://github.com/Infinisil/nixpkgs/tree/feat/warn-no-type

@dasJ
Copy link
Member Author

dasJ commented Dec 7, 2021

@infinisil Do you want to open a PR yourself or do you want me to pick the commit?

@infinisil
Copy link
Member

Feel free to pick the commit :)

@dasJ dasJ force-pushed the feat/warn-no-type branch 2 times, most recently from 48b2ac7 to 4d71487 Compare December 7, 2021 17:04
@dasJ
Copy link
Member Author

dasJ commented Dec 7, 2021

After talking to @domenkozar we replaced the error message with something that is actionable and more clear

@dasJ dasJ force-pushed the feat/warn-no-type branch 2 times, most recently from ed52ff1 to 2767b73 Compare December 9, 2021 17:01
@github-actions github-actions bot added the 8.has: module (update) This PR changes an existing module in `nixos/` label Dec 9, 2021
@infinisil
Copy link
Member

Hmm, I'm having second thoughts. With this we're essentially telling people to always specify a type, but at least one of the main reasons we want to do this is that the fallback type of types.unspecified has bad behavior (similar to types.attrs).

An alternative to fix this problem is to change the default type to something that doesn't have bad behavior, like types.anything from #97119. Then it would not be a problem to specify no type.

@infinisil
Copy link
Member

I guess another reason for wanting good types is documentation and actual type checking (duh!). Though the cost of requiring every option definition having a type is significant. I suspect there to be many many instances of people declaring their own options without types, me including.

@dasJ
Copy link
Member Author

dasJ commented Dec 9, 2021

It's the same logic (I think) as with the description which throws a warning as well without the dangers of changing the default type.
It shouldn't change anything for anyone with custom modules (us included) but fail ofborg evaluation because ofborg expects the eval to have no warnings. So this should in the end have no effect for no-nixpkgs projects apart from the warnings.
We could implement an option to disable these warnings (if it annoys enough people in the future) that could also disable the warnings about the descriptions.

@infinisil
Copy link
Member

The PR in its current state throws warnings for user-defined options without a type. E.g. with a config.nix like

{ lib, config, ... }: {
  options.foo = lib.mkOption {
    default = "test";
  };

  config = {
    fileSystems."/".device = "/nodev";
    boot.loader.grub.device = "nodev";

    environment.etc.foo.text = config.foo;
  };
}

Evaluating it:

$ nix-instantiate nixos --arg configuration ./config.nix -A system
trace: warning: Option `foo' defined in `/home/infinisil/src/nixpkgs/config.nix' has no type.
       Please specify a valid types from <nixpkgs/lib/types.nix> or use types.unspecified if there is no appropriate type.
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/iw5i9w8z1z0klb8k9ja5y4iqh4bcy7d0-nixos-system-nixos-22.05pre-git.drv

@dasJ
Copy link
Member Author

dasJ commented Dec 9, 2021

Yeah I wasn't arguing that, but I think we could make the warning an opt-out when people get annoyed by it

@infinisil
Copy link
Member

My point was that not setting a description doesn't give any warnings for user-defined options, in contrast to an unset type.

I think if we just throw warnings for undeclared types in nixpkgs (or rather, for all modules that are rendered into the manual, which can include user modules if documentation.nixos.includeAllModules is enabled) should be good enough, which is also what happens for description. I don't think we should give warnings for other options, especially as we will have a better default type eventually with types.anything.

For the suggestion of only giving a warning for rendered options, here's a patch that makes it work, notably assuming that types.unspecified is an unspecified type, which honestly sounds like it's original intention:

diff --git a/lib/options.nix b/lib/options.nix
index 5d52f065af0..c7970d38e73 100644
--- a/lib/options.nix
+++ b/lib/options.nix
@@ -185,7 +185,13 @@ rec {
             then true
             else opt.visible or true;
           readOnly = opt.readOnly or false;
-          type = opt.type.description or null;
+          type =
+            if opt.type.name == "unspecified"
+            then lib.warn
+              ("Option `${showOption loc}' defined in ${showFiles opt.declarations} has no type.\n"
+              +"       Please specify a valid types from <nixpkgs/lib/types.nix> or use types.raw if there is no appropriate type.")
+              null
+            else opt.type.description or null;
         }
         // optionalAttrs (opt ? example) { example = scrubOptionValue opt.example; }
         // optionalAttrs (opt ? default) { default = scrubOptionValue opt.default; }

This notably does mention types.raw from #132448 as the type to use when no other type is appropriate, because types.unspecified wouldn't work of course. We could also mention types.anything here as an alternative. Differences are:

  • types.raw doesn't recurse into the structure at all, also meaning it doesn't do any merging and only one definition is allowed. This should be used for things like package sets, e.g. services.xserver.windowManager.xmonad.haskellPackages
  • types.anything fully recurses into the structure with a safe default merge function that doesn't discard anything, it supports nested mkIf's and co. which get properly handled.

@dasJ dasJ force-pushed the feat/warn-no-type branch 2 times, most recently from ec3f749 to 55dce12 Compare December 10, 2021 12:31
@infinisil
Copy link
Member

I created #162271 as an updated version of this, along with some improvements

@dasJ dasJ closed this Mar 1, 2022
@dasJ dasJ deleted the feat/warn-no-type branch March 1, 2022 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10
Projects
None yet
Development

Successfully merging this pull request may close these issues.

All options should have types
3 participants