pipx install doesn't prompt for credentials for private vcs url

[alexkillen]

Describe the bug
Regular pipx install command doesn't work for private vcs urls that require authentication. Running pipx install --spec git+https://<some_private_repo>.git <package_name> does not prompt the user for username and password. If the --verbose option is provided, it works fine.

How to reproduce
Run above command using a private vcs url (e.g. github, gitlab, bitbucket) and do not use --verbose.

Expected behavior
Should prompt user for credentials, even when --verbose is not used.

Environment

$ python3 --version
Python 3.6.8
$ pipx --version
0.14.0.0
$ head -n 2 /etc/os-release
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
$ uname -srm
Linux 4.15.0-64-generic x86_64

[cs01]

My guess is that this is because the pipx animation takes over the terminal and hides the prompt for the password. The animation runs until the subprocess command finishes. Does it just animate forever and never finish?

The fix would be to not do the animation. Ways to do this:

• Use verbose flag as you discovered
• Add flag or env var to pipx to suppresses animation
• Never show animation
• Document problem and workaround in cli help text

[alexkillen]

Yes, it animates forever and never completes. As for a solution, a good compromise might be to disable animations when the --spec argument is used. If that's not an option I'd lean towards a --no-anims type flag or env var to disable animations, as using --verbose is not that intuitive, I only discovered that worked while trying to debug the issue.

It would of course be nice if pipx could detect when user prompts occur and drop out of the animation when they do, but that sounds like it would require considerable effort.

btw, pipx is great, the Python ecosystem has needed this for years, too many people try to force pip and pypi into this role and it causes a lot of confusion. Keep up the great work.

[cs01]

@uranusjr how does pipenv solve this?

[alexkillen]

Had a quick check through the pipenv source and it looks like it's using this: https://github.com/sarugaku/vistir#spinner

[cs01]

Thanks!

What happens when installing from a protected source with pipenv? Does it spin indefinitely or does it stop spinning and drop back to the user/password prompt? If it drops back to the password prompt I'll take a look around the visitr source code to see how it does it. I don't want to add it as a dependency because it depends on quite a few things, and I would like to keep pipx as dependency-free as possible.

[cs01]

I wonder if writing to a different stream might fix the issue too. Pipx writes the spinner to stderr. If pip writes to stderr, having pipx do the animation on stdout might fix it.

https://github.com/pipxproject/pipx/blob/master/pipx/animate.py

[uranusjr]

@cs01 Pipenv doesn’t have a solution either; it requires the credentials before installation (using environment variable injection). We have yet to find a good way to interact with pip.

[alexkillen]

When I install from a private VCS using pipenv, it does actually prompt me for username and password, but the spinner still shows and keeps showing wherever the cursor is, leaving behind a trail of individual animation frames of the spinner, so it looks quite odd and isn't really an ideal solution.

I think since usage of a private VCS isn't really going to be widespread, going with the simplistic approach of having a flag to disable animations and a section in the docs about using it with private VCS URLs might be the best "solution" for the moment. Unless you find that changing the animation stream works of course.

[itsayellow]

I just ran into this recently during #302 , when I just deleted one character from my public github repo name, in order to try and force an error. Instead of getting an error that there was no package there, I got a github username prompt next to the animation spinner. It was confusing exactly what was going on.

The only way to get out of this was Ctrl-C. It seems like at least maybe if we have no good way of the user entering username/password we should find a way to disallow it, or to exit early when we see it.

[cs01]

Pipenv doesn’t have a solution either; it requires the credentials before installation (using environment variable injection). We have yet to find a good way to interact with pip.

We could try to use environment variables like pipenv, though @alexkillen's comment indicates pipenv doesn't have behavior any better than pipx, at least in some circumstances.

Another thought is maybe pip detects whether stdout/stderr is a tty and modifies its behavior based on that. Oftentimes programs will have different interactive/non-interactive based on whether the file descriptor is a tty. If we make all of pipx's calls to pip redirect to a file, then pip will not be writing to a tty and maybe it will abort immediately when it reaches a password prompt.

[uranusjr]

For git+ (and other VCS) specifically pip does little than forwarding the URL to the VCS command, so I think the problem is in Git (or even more specifically the credential helper).

For regular HTTP(S) access pip doesn’t do it that well either, see pypa/pip#2920.

[cs01]

It looks like pip's undocumented --no-input flag may do what we're looking for pypa/pip#2920 (comment).

[venaturum]

This is also an issue for private package feeds requiring credentials, eg an Azure artifacts feed

It neither prompts, or works with artifacts-keyring like pip does

[itsayellow]

I wasn't aware of artifacts-keyring. It looks like you need to install artifacts-keyring into the same environment as pip. This likely doesn't work with pipx because we use our own shared venv to host our pip, and it is separate from the stock system python environment (user or system).

[mithun]

I'm running into a similar issue when trying to install from a private registry or index-url (e.g., github/gitlab/artifactory/etc) that requires authentication.

In my case, I'm using keyring to store the credentials required, and pip will automatically use keyring if available. A typical pip based workflow would be python -m venv .venv && source .venv/bin/activate && pip install keyring && pip install -i https://authenticated.example.com/pypi/simple some-package-name.

In order to achieve something similar with pipx, pipx should have a mechanism to (pre?) inject packages (keyring, artifacts-keyring, etc) into an app's venv before the app itself is installed.

[MrPointer]

It looks like pip's undocumented --no-input flag may do what we're looking for

@cs01 I'm not so sure... You'd still need to enter keyring's password somehow, don't you?
Besides, I've given it a try with the plain-text keyring backend, which doesn't prompt for a password, and it just fails silently...

Seems to me that pipx should add support for input, just as pip itself would do. It's not an easy solution though, I'm sure about it...

[uranusjr]

If you're so sure I highly recommend drafting a PR, discussion can be more constructive if it is really that simple :)

[MrPointer]

@uranusjr Oh God, I accidentally left out the word "not" 😂
Definitely not easy!!

[uranusjr]

Internet is so hard 😥

[cs01]

Perhaps the ‘GIT_ASKPASS‘ environment variable is what we’re looking for here.

https://git-scm.com/docs/gitcredentials

[MrPointer]

@cs01 Actually, in my case it's failing to authenticate against a private Nexus repository, no git is involved.
So while GIT_ASKPASS might be the solution for git repos, it doesn't seem to be a generic one.

[Darsstar]

In order to achieve something similar with pipx, pipx should have a mechanism to (pre?) inject packages (keyring, artifacts-keyring, etc) into an app's venv before the app itself is installed.

#829 is a draft PR that adds something that sort of does that

[martina-oefelein]

As a workaround, one can include the username:password or personal access token in the package in the package index URL, see PIP documentation:

 PIP_EXTRA_INDEX_URL=https://username:[email protected]/simple pipx install my-package

[Darsstar]

Pip 23.1 allows you to configure it so it will use the keyring library even when --no-input is used. Now that that is possible I am of the opinion that Pipx should use --no-input unless --verbose is passed.

Alternatively passing -qqqqq and --log to Pip and a more complicated animation implementation might also be a direction that can be explored.

I am working on a PR for --no-input strategy. The test are giving me some trouble on Python < 3.11, so feel free to nerd-snipe me!