`pyenv-build` calls `conda install --yes pip` and updates `conda` to a different version than desired #2070

humitos · 2021-09-20T10:54:38Z

Hi all! Thanks for this amazing project.

I found a problem with conda version not being the desired one. This happens because after conda is installed pyenv-build calls conda install --yes pip at this line https://github.com/pyenv/pyenv/blob/v2.0.6/plugins/python-build/bin/python-build#L1037 which makes conda to updates all the packages installed in the base system:

...

The following packages will be UPDATED:

  ...
  conda                                       4.7.12-py37_0 --> 4.10.3-py37h06a4308_0
  ...

...

Note that conda-4.7.12 was updated to conda-4.10.3 but that's not version that I told pyenv-build to install.

I'd suggest to not install pip (why pip is installed by default?) or find a way to tell conda to not update other dependencies (I tried passing --no-update-deps but it didn't work as expected).

Let me know if I can be useful here.

Click to see the full output

▶ asdf install python miniconda3-4.7.12
python-build miniconda3-4.7.12 /home/humitos/.asdf/installs/python/miniconda3-4.7.12
Downloading Miniconda3-4.7.12-Linux-x86_64.sh...
-> https://repo.anaconda.com/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh
Installing Miniconda3-4.7.12-Linux-x86_64...
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/humitos/.asdf/installs/python/miniconda3-4.7.12

  added / updated specs:
    - pip


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2021.7.5   |       h06a4308_1         113 KB
    certifi-2021.5.30          |   py37h06a4308_0         139 KB
    conda-4.10.3               |   py37h06a4308_0         2.9 MB
    openssl-1.1.1l             |       h7f8727e_0         2.5 MB
    pip-21.0.1                 |   py37h06a4308_0         1.8 MB
    ------------------------------------------------------------
                                           Total:         7.4 MB

The following packages will be UPDATED:

  ca-certificates                               2019.8.28-0 --> 2021.7.5-h06a4308_1
  certifi                                  2019.9.11-py37_0 --> 2021.5.30-py37h06a4308_0
  conda                                       4.7.12-py37_0 --> 4.10.3-py37h06a4308_0
  openssl                                 1.1.1d-h7b6447c_2 --> 1.1.1l-h7f8727e_0
  pip                                         19.2.3-py37_0 --> 21.0.1-py37h06a4308_0



Downloading and Extracting Packages
conda-4.10.3         | 2.9 MB    | ################################################################################################### | 100% 
certifi-2021.5.30    | 139 KB    | ################################################################################################### | 100% 
ca-certificates-2021 | 113 KB    | ################################################################################################### | 100% 
pip-21.0.1           | 1.8 MB    | ################################################################################################### | 100% 
openssl-1.1.1l       | 2.5 MB    | ################################################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installed Miniconda3-4.7.12-Linux-x86_64 to /home/humitos/.asdf/installs/python/miniconda3-4.7.12

NOTE that I'm using pyenv-build via asdf (https://asdf-vm.com/)

The text was updated successfully, but these errors were encountered:

astrojuanlu · 2021-09-20T11:12:13Z

Left a comment upstream conda/conda#7959 (comment)

native-api · 2021-09-20T11:15:45Z

Pip is installed because users typically expect it to be available.

I'm not sure what we can do from our side other than re-align Pyenv's Anaconda entries and naming with their practices.

What is Anaconda's autoupdate policy? Does it allow using old package versions (like vanilla CPython), or it enforces everyone to use the latest versions within some version envelope? Does it autoupdate Python, too -- if yes, to what extent?

If e.g. it feels free to update anything as long as Python's major.minor stays the same, we'll have to only have one Anaconda/Miniconda entry per Python's major.minor and rename them appropriately.

anton-petrov · 2021-09-21T20:16:02Z

Hi @humitos ! Unfortunately, we can't help you, because this is a bug of conda, not pyenv, as mentoined in comment, that you share:

The --no-update-deps flag is completely ignored

astrojuanlu · 2021-09-22T04:51:38Z

IMO despite the unfortunate conda bug, pyenv could still look for a workaround. For example, something like conda install --yes pip "conda=$CURRENT_VERSION". And in any case, the issue exists, regardless of the upstream situation.

humitos · 2021-09-22T08:22:06Z

@anton-petrov

Unfortunately, we can't help you, because this is a bug of conda, not pyenv, as mentoined in comment, that you share:

I think there are two different problems here:

pyenv user not getting the version they are told pyenv to install (this can be solved by pyenv project)
conda upgrading all system packages when told just to install a new one (this can be solved by conda and it's outside the scope of pyenv)

I still think that pyenv could do something to solve its own problem. One suggestion would be what @astrojuanlu said in the previous comment.

Another possible solution would be just to not install pip by default and leaves that task to the user. Actually, pyenv could read an evironment variable to decide if install pip or not: PYENV_INSTALL_PIP=false would be enough for my usage case

Finally, pyenv could also add --no-update-deps and relies on conda to solve their issue.

@native-api

Pip is installed because users typically expect it to be available.

While I tend to agree here, I'd say that this is more common on regular Python users, but I don't have the same feeling for conda users since packages are installed via conda install instead of pip.

Thank you all!

humitos · 2021-09-22T08:51:53Z

For example, something like conda install --yes pip "conda=$CURRENT_VERSION".

I can confirm this solution works:

...
  added / updated specs:
    - conda=4.7.12
    - pip

...

The following packages will be UPDATED:

  ca-certificates                               2019.8.28-0 --> 2021.7.5-h06a4308_1
  certifi                                  2019.9.11-py37_0 --> 2021.5.30-py37h06a4308_0
  openssl                                 1.1.1d-h7b6447c_2 --> 1.1.1l-h7f8727e_0
  pip                                         19.2.3-py37_0 --> 21.0.1-py37h06a4308_0

...

Note the conda package is not going to be updated.

This is the diff that I use to make this work:

diff --git a/plugins/python-build/bin/python-build b/plugins/python-build/bin/python-build
index a18680a4..ea976328 100755
--- a/plugins/python-build/bin/python-build
+++ b/plugins/python-build/bin/python-build
@@ -1042,7 +1042,9 @@ build_package_anaconda() {
 
 build_package_miniconda() {
   build_package_anaconda "$@"
-  "${PREFIX_PATH}/bin/conda" install --yes "pip"
+  # Workaround to not upgrade conda when installing pip
+  # see https://github.com/pyenv/pyenv/issues/2070
+  "${PREFIX_PATH}/bin/conda" install --yes "pip" "conda=$(${PREFIX_PATH}/bin/conda --version | cut -d ' ' -f 2)"
 }
 
 build_package_copy() {

Click to see full output

▶ asdf install python miniconda3-4.7.12  
python-build miniconda3-4.7.12 /home/humitos/.asdf/installs/python/miniconda3-4.7.12
Downloading Miniconda3-4.7.12-Linux-x86_64.sh...
-> https://repo.anaconda.com/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh
Installing Miniconda3-4.7.12-Linux-x86_64...
Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.7.12
  latest version: 4.10.3

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: /home/humitos/.asdf/installs/python/miniconda3-4.7.12

  added / updated specs:
    - conda=4.7.12
    - pip


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2021.7.5   |       h06a4308_1         113 KB
    certifi-2021.5.30          |   py37h06a4308_0         139 KB
    openssl-1.1.1l             |       h7f8727e_0         2.5 MB
    pip-21.0.1                 |   py37h06a4308_0         1.8 MB
    ------------------------------------------------------------
                                           Total:         4.5 MB

The following packages will be UPDATED:

  ca-certificates                               2019.8.28-0 --> 2021.7.5-h06a4308_1
  certifi                                  2019.9.11-py37_0 --> 2021.5.30-py37h06a4308_0
  openssl                                 1.1.1d-h7b6447c_2 --> 1.1.1l-h7f8727e_0
  pip                                         19.2.3-py37_0 --> 21.0.1-py37h06a4308_0



Downloading and Extracting Packages
certifi-2021.5.30    | 139 KB    | ############################################################################################################################################################################################ | 100% 
ca-certificates-2021 | 113 KB    | ############################################################################################################################################################################################ | 100% 
pip-21.0.1           | 1.8 MB    | ############################################################################################################################################################################################ | 100% 
openssl-1.1.1l       | 2.5 MB    | ############################################################################################################################################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installed Miniconda3-4.7.12-Linux-x86_64 to /home/humitos/.asdf/installs/python/miniconda3-4.7.12

Time: 0h:00m:18s                                                                                                                                                                                                                       

~                                                                                                                                                                                                                                     
▶ asdf shell python miniconda3-4.7.12    

~                                                                                                                                                                                                                                     
▶ conda --version
conda 4.7.12

native-api · 2021-09-22T15:54:35Z

We can do this... but won't this just postpone the inevitable? Whenver you run conda install the next time, everything will be updated again.

The real solution -- if we're not going to change the conda packaging scheme -- seems to be to configure Anaconda to not autoupdate itself: https://stackoverflow.com/questions/48726621/is-it-possible-to-lock-versions-of-packages-in-anaconda

conda config --set auto_update_conda False

humitos · 2021-09-22T16:54:43Z

@native-api

We can do this... but won't this just postpone the inevitable? Whenver you run conda install the next time, everything will be updated again.

Maybe. However, there are two benefits still:

the user gets the conda version that asked pyenv for
the user can confirm/cancel the upgrade of conda when installing any other dependency --they will be shown with a dialog with a summary to confirm/cancel and can take the action they think is best.

The real solution -- if we're not going to change the conda packaging scheme -- seems to be to configure Anaconda to not autoupdate itself

Setting conda to not auto update itself makes sense to me. That's basically what the user wants when calling pyenv install miniconda3-4.7.10, right? Otherwise, the user would have selected the latest version.

native-api · 2021-09-22T19:45:17Z

Setting conda to not auto update itself makes sense to me. That's basically what the user wants when calling pyenv install miniconda3-4.7.10, right? Otherwise, the user would have selected the latest version.

So, the autoupdate shouldn't be disabled in the *conda*-latest scripts, right?

native-api · 2021-09-22T19:47:50Z

@anton-petrov This looks like not a bug but an intentional feature to me, especially in the light of the aforementioned dedicated setting to control it. Whatever we think of this feature, we need to decide what to do about it.

anton-petrov · 2021-09-23T09:25:05Z

@native-api I think we can use a temporary workaround for this from #2070 (comment)
and in the near future, implement this functionality:

... pyenv could also add --no-update-deps and relies on conda to solve their issue.

Just created a PR #2074

When installing miniconda3, `pyenv-build` calls: conda install --yes pip which automatically updates conda to its latest version and making us lossing our pinned version. This commit adds a patch to keep installing pip after conda install, but pinning conda to the current version to avoid upgrading it automatically. See pyenv/pyenv#2070

native-api · 2021-10-05T02:35:20Z

I've implemented the auto_update_conda option.

@humitos See the CAUTION note there -- it turns out, it's still not bulletproof.

astrojuanlu mentioned this issue Sep 20, 2021

--no-update-deps is not respected conda/conda#7959

Open

native-api added third-party the problem is in third-party software type: enhancement labels Sep 20, 2021

humitos mentioned this issue Sep 21, 2021

New config for new docker build images readthedocs/readthedocs.org#8478

Merged

anton-petrov closed this as completed Sep 21, 2021

native-api reopened this Sep 22, 2021

anton-petrov mentioned this issue Sep 23, 2021

Don't update conda when installing pip #2074

Merged

4 tasks

anton-petrov closed this as completed in #2074 Sep 23, 2021

native-api mentioned this issue Oct 5, 2021

Disable conda package self-update #2094

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`pyenv-build` calls `conda install --yes pip` and updates `conda` to a different version than desired #2070

`pyenv-build` calls `conda install --yes pip` and updates `conda` to a different version than desired #2070

humitos commented Sep 20, 2021

astrojuanlu commented Sep 20, 2021

native-api commented Sep 20, 2021 •

edited

Loading

anton-petrov commented Sep 21, 2021

astrojuanlu commented Sep 22, 2021

humitos commented Sep 22, 2021

humitos commented Sep 22, 2021

native-api commented Sep 22, 2021

humitos commented Sep 22, 2021

native-api commented Sep 22, 2021

native-api commented Sep 22, 2021

anton-petrov commented Sep 23, 2021

native-api commented Oct 5, 2021

pyenv-build calls conda install --yes pip and updates conda to a different version than desired #2070

pyenv-build calls conda install --yes pip and updates conda to a different version than desired #2070

Comments

humitos commented Sep 20, 2021

astrojuanlu commented Sep 20, 2021

native-api commented Sep 20, 2021 • edited Loading

anton-petrov commented Sep 21, 2021

astrojuanlu commented Sep 22, 2021

humitos commented Sep 22, 2021

humitos commented Sep 22, 2021

native-api commented Sep 22, 2021

humitos commented Sep 22, 2021

native-api commented Sep 22, 2021

native-api commented Sep 22, 2021

anton-petrov commented Sep 23, 2021

native-api commented Oct 5, 2021

`pyenv-build` calls `conda install --yes pip` and updates `conda` to a different version than desired #2070

`pyenv-build` calls `conda install --yes pip` and updates `conda` to a different version than desired #2070

native-api commented Sep 20, 2021 •

edited

Loading