Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib.cli.escapeShellArg{,s}: Only escape when necessary #333744

Merged
merged 6 commits into from
Aug 16, 2024

Conversation

Gabriella439
Copy link
Contributor

@Gabriella439 Gabriella439 commented Aug 10, 2024

These utilities will now leave the string undisturbed if it doesn't need to be quoted (because it doesn't have any special characters). This can help generate nicer-looking command lines.

Description of changes

Things done


Add a 👍 reaction to pull requests you find important.

These utilities are like `escapeShellArg{,s}` except that they will
leave the string undisturbed if it doesn't need to be quoted (because
it doesn't have any special characters).  This can help generate
nicer-looking command lines.
@github-actions github-actions bot added the 6.topic: lib The Nixpkgs function library label Aug 10, 2024
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Aug 10, 2024
@infinisil
Copy link
Member

Oh this is nice!

I don't think we need a separate function for this actually. While the existing ones have Shell in their name, they actually specify bash in the description:

escapeShellArg: Quote string to be used safely within the Bourne shell.

And even if we assume escapeShellArg to be POSIX shell specific, it's likely that almost (if not exactly) the same characters need escaping as with bash.

@infinisil
Copy link
Member

Also a minor nit for the commit message: This function is not in lib.cli but rather lib.strings (though it really should be in the former).

lib/strings.nix Outdated Show resolved Hide resolved
@Gabriella439
Copy link
Contributor Author

I'll get to your feedback soon. Also, just to clarify: are you saying it's okay to make this the new behavior of escapeShellArg?

@infinisil
Copy link
Member

I'll get to your feedback soon. Also, just to clarify: are you saying it's okay to make this the new behavior of escapeShellArg?

Yes I think that would be okay :)

@ofborg ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild 10.rebuild-linux-stdenv This PR causes stdenv to rebuild 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Aug 12, 2024
@emilazy
Copy link
Member

emilazy commented Aug 13, 2024

This would need to target staging now due to all the rebuilds.

@Gabriella439 Gabriella439 changed the base branch from master to staging August 15, 2024 22:28
@Gabriella439 Gabriella439 changed the title lib.cli: Add escapeBashArg{,s} lib.cli.escapeShellArg{,s}: Only escape when necessary Aug 15, 2024
@Gabriella439
Copy link
Contributor Author

Alright, I think this is ready for re-review (and rebased against staging)

@Gabriella439 Gabriella439 requested a review from infinisil August 16, 2024 01:23
Copy link
Member

@emilazy emilazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Disadvantaging non‐ASCII alphabetical characters is unfortunate, but probably better than trying to correctly handle Unicode in a language that doesn’t really support it. This seems like it should be a big improvement for standard CLI cases. Hard to say if there’ll be any Hyrum’s law type breakage, but I think it’s worth a try at least.

I’ll let @infinisil do another round of reviews, but could you squash the commits together before merge? Right now this PR would break git bisect.

@Gabriella439
Copy link
Contributor Author

@emilazy: I typically squash merge (and clean up the commit message in the process of doing so)

Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

lib/strings.nix Outdated Show resolved Hide resolved
NickCao pushed a commit that referenced this pull request Aug 24, 2024
This needs to be a single‐quoted string; #333744 broke this incorrect
use. Hyrum’s law strikes again!
@sersorrel
Copy link
Contributor

This also caused a regression in the dokuwiki module.

@emilazy
Copy link
Member

emilazy commented Sep 7, 2024

escapeShellArg was not at all correct for PHP even before this, so the DokuWiki module should be fixed.

rycee added a commit to nix-community/home-manager that referenced this pull request Sep 8, 2024
rycee added a commit to nix-community/home-manager that referenced this pull request Sep 8, 2024
rycee added a commit to nix-community/home-manager that referenced this pull request Sep 8, 2024
rycee added a commit to nix-community/home-manager that referenced this pull request Sep 8, 2024
rycee added a commit to nix-community/home-manager that referenced this pull request Sep 8, 2024
@emilazy
Copy link
Member

emilazy commented Sep 10, 2024

Another Hyrum’s law issue: LnL7/nix-darwin#1068.

(I don’t think it was necessarily wrong to do the PR this way or anything; just feel it is interesting/useful have a reference for the kinds of ways semantics‐preserving changes like this can result in downstream users breaking.)

emilazy pushed a commit that referenced this pull request Sep 10, 2024
PHP strings don't obey shell quoting rules. See #333744.
@Gabriella439
Copy link
Contributor Author

Yeah, this is why my original change didn't make this behavior the default.

PerchunPak pushed a commit to PerchunPak/home-manager that referenced this pull request Sep 17, 2024
brckd pushed a commit to brckd/home-manager that referenced this pull request Sep 24, 2024
Mikilio pushed a commit to Mikilio/home-manager that referenced this pull request Oct 11, 2024
roberth added a commit to hercules-ci/hercules-ci-effects that referenced this pull request Oct 28, 2024
YorikSar added a commit to YorikSar/nix-darwin that referenced this pull request Nov 4, 2024
Changes to escapeShellArg introduced in
NixOS/nixpkgs#333744 made different versions of
nixpkgs behave differently. If current nix-darwin is used with nixpkgs
before that change, labels end up having labels quoted twice
(see LnL7#1085), but without
changes from LnL7#1055, with new
nixpkgs, labels end up not quoted at all, and ShellCheck ends up
complaining that commas might have been used as array item separator
(see https://www.shellcheck.net/wiki/SC2054).

Use the old version of escapeShellArg to always escape the list of
labels and make nix-darwin work with both old and new versions of
nixpkgs.

Fixes LnL7#1085
YorikSar added a commit to YorikSar/nix-darwin that referenced this pull request Nov 7, 2024
Changes to escapeShellArg introduced in
NixOS/nixpkgs#333744 made different versions of
nixpkgs behave differently. If current nix-darwin is used with nixpkgs
before that change, labels end up having labels quoted twice
(see LnL7#1085), but without
changes from LnL7#1055, with new
nixpkgs, labels end up not quoted at all, and ShellCheck ends up
complaining that commas might have been used as array item separator
(see https://www.shellcheck.net/wiki/SC2054).

Use the old version of escapeShellArg to always escape the list of
labels and make nix-darwin work with both old and new versions of
nixpkgs.

Fixes LnL7#1085
YorikSar added a commit to YorikSar/nix-darwin that referenced this pull request Nov 7, 2024
Changes to escapeShellArg introduced in
NixOS/nixpkgs#333744 made different versions of
nixpkgs behave differently. If current nix-darwin is used with nixpkgs
before that change, labels end up having labels quoted twice
(see LnL7#1085), but without
changes from LnL7#1055, with new
nixpkgs, labels end up not quoted at all, and ShellCheck ends up
complaining that commas might have been used as array item separator
(see https://www.shellcheck.net/wiki/SC2054).

Use the old version of escapeShellArg to always escape the list of
labels and make nix-darwin work with both old and new versions of
nixpkgs.

Fixes LnL7#1085
iFreilicht added a commit to nix-community/disko that referenced this pull request Nov 11, 2024
Regression introduced in 94bc0f5,
because I developed against unstable, which has
NixOS/nixpkgs#333744 merged, while 24.05
doesn't.

Partly shellchecks mistake, see
koalaman/shellcheck#2891 but whatever, we
can't do much more about it than ignore the warning.

Fixes #868
iFreilicht added a commit to nix-community/disko that referenced this pull request Nov 13, 2024
Regression introduced in 94bc0f5,
because I developed against unstable, which has
NixOS/nixpkgs#333744 merged, while 24.05
doesn't.

Partly shellchecks mistake, see
koalaman/shellcheck#2891 but whatever, we
can't do much more about it than ignore the warning.

Fixes #868
Noodlez1232 pushed a commit to Noodlez1232/home-manager that referenced this pull request Nov 21, 2024
@ncfavier
Copy link
Member

Hard to say if there’ll be any Hyrum’s law type breakage

Yup: #367288

@emilazy
Copy link
Member

emilazy commented Dec 28, 2024

Sorry for not pushing back harder on repurposing the name here. I think it’s pointless to revert at this point now that it’s in a stable release though (but it’s a useful lesson for next time).

colonelpanic8 pushed a commit to colonelpanic8/home-manager that referenced this pull request Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants