[Win] Windows wheels not available on pypi for 5.9.4

[chrisoro]

Summary

OS: Windows
Architecture: 64bit, 32bit
Psutil version: 5.9.4
Python version: python3
Type: wheels

Description

Same problem as in 5.9.2 release (see #2134). Most wheels for Windows are missing.
Please take the time and check the deploy job to make sure it doesn't happen in the future. It's super inconvenient, especially if one uses "psutil~=5.9.0" as a requirement.

[giampaolo]

What error are you getting? Please paste pip install psutil output.

[blthayer]

This isn't just an issue for windows wheels. Wheels for most Python versions are missing.

Take a look at the files for 5.9.4. There are wheels for Python 2.7, 3.6, and a Mac wheel for 3.8.

By contrast, the files for 5.9.3 have wheels for many platform/OS and Python version combinations.

When attempting to install the latest version of psutil via poetry, this results in the following error:

...
  RuntimeError

  Unable to find installation candidates for psutil (5.9.4)
...

(because there's no Linux wheel for Python 3.10 in my case)

[giampaolo]

Wheels for most Python versions are missing.

That's because we recently added wheels supporting multiple python versions, see #2102. Please paste the full output of pip install -v psutil and/or poetry. Also what's your python version? (python3 -V)

[blthayer]

Thanks @giampaolo for linking to that issue - today I learned about ABI3.

Here's what I've found so far:

The ABI3 wheel works fine on my x86_64 machine running Ubuntu 22.04 and Python 3.10.8:

bthayer@zbook:~/tmp$ python --version
Python 3.10.8
bthayer@zbook:~/tmp$ python -m venv venv
bthayer@zbook:~/tmp$ source venv/bin/activate
(venv) bthayer@zbook:~/tmp$ python -m pip install "psutil==5.9.4"
Collecting psutil==5.9.4
  Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 280.2/280.2 kB 5.2 MB/s eta 0:00:00
Installing collected packages: psutil
Successfully installed psutil-5.9.4

[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
(venv) bthayer@zbook:~/tmp$ python
Python 3.10.8 (main, Oct 12 2022, 12:54:48) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'x86_64'
>>> exit()

However, on my ARM64 (aka aarch64) machine running Ubuntu 20.04 (also Python 3.10.8), it has to install from source (sorry for the clutter - I hit Ctrl+C accidentally/reactively because I forgot to specify the exact version for psutil):

bthayer@sting:~/tmp$ python --version
Python 3.10.8
bthayer@sting:~/tmp$ python -m venv venv
bthayer@sting:~/tmp$ source venv/bin/activate
(venv) bthayer@sting:~/tmp$ python -m pip install psutil
Collecting psutil
  Downloading psutil-5.9.4.tar.gz (485 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 485.8/485.8 kB 2.3 MB/s eta 0:00:00
^C  Installing build dependencies ... canceled
ERROR: Operation cancelled by user

[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
(venv) bthayer@sting:~/tmp$ python -m pip install "psutil==5.9.4"
Collecting psutil==5.9.4
  Using cached psutil-5.9.4.tar.gz (485 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: psutil
  Building wheel for psutil (pyproject.toml) ... done
  Created wheel for psutil: filename=psutil-5.9.4-cp310-abi3-linux_aarch64.whl size=293317 sha256=ee82d6b26f766d95e9eb92243aff1abdcf70ab19500b3da4b897e4a5f987addb
  Stored in directory: /home/bthayer/.cache/pip/wheels/8c/f0/2c/0d7d6e64319efa308609a6844a955ef94aa797388843265de0
Successfully built psutil
Installing collected packages: psutil
Successfully installed psutil-5.9.4

[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
(venv) bthayer@sting:~/tmp$ python
Python 3.10.8 (main, Oct 18 2022, 11:26:55) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'aarch64'

This isn't ideal for my use case as we deploy to ARM64 devices, and it's preferred to use pre-built wheels when possible.

I'll see if I can create a minimum working example of the failure and report back (I can't just paste output from my project without obfuscation).

PS - thank you for psutil! It's a wonderful package 😄

[giampaolo]

Ok, that's because we currently are not distributing aarch64 wheels. There's another (or multiple) ticket for that so I'm closing this as a duplicate. :)

[blthayer]

Okay, on the poetry side of things the symptoms are the same: seems to install from wheel for my x86_64 machine, and seems to install from source on my aarch64 machine.

The reason I was getting the Unable to find installation candidates for psutil (5.9.4) error earlier seems to be because our internal artifact server for whatever reason is not storing the source distribution. So when I attempt to install via poetry on ARM64 using our internal artifact server it fails because there's neither a wheel nor source distribution:

Updating dependencies
Resolving dependencies... (3.0s)

Writing lock file

Finding the necessary packages for the current system

Package operations: 1 install, 0 updates, 0 removals

  • Installing psutil (5.9.4): Failed

  RuntimeError

  Unable to find installation candidates for psutil (5.9.4)

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/chooser.py:103 in choose_for
       99│ 
      100│             links.append(link)
      101│ 
      102│         if not links:
    → 103│             raise RuntimeError(f"Unable to find installation candidates for {package}")
      104│ 
      105│         # Get the best link
      106│         chosen = max(links, key=lambda link: self._sort_key(package, link))
      107│ 

Was the exclusion of ARM64 wheel in 5.9.4 intentional?

[blthayer]

@giampaolo - jinx! Commenting at the same time 😄

Okay - good to know you aren't distributing aarch64 wheels anymore. I'll say those are very useful for my company and use case, but we can make due with a source distribution (I just have to tinker with our damned artifact server).

[giampaolo]

Look at INSTALL.rst. It tells what dependencies to install on Linux to compile from sources.

[chrisoro]

@giampaolo sorry for the laste response, I didn't find the time to answer. Looking at our logs and the date of file upload some wheels have been missing at the initial release on Nov 7th. They have been added on Nov 8th and are fine now.
Namely psutil-5.9.4-cp36-abi3-win_amd64.whl and psutil-5.9.4-cp36-abi3-win32.whl