[python3] Upgrade to 3.9.0

[Hoikas]

This upgrades the python3 port to 3.9.0. Per #13741, configuring python failed on macOS Big Sur beta. According to the python release notes, that particular bug was fixed in 3.8.4 rc1. However, complete support for Big Sur was not promised until 3.9.0. Therefore we skip the bugfix updates to the 3.8 series and go directly to 3.9.0.

• What does your PR fix? Fixes [python3] build failure #13741

• Which triplets are supported/not supported? Have you updated the CI baseline? No change.

• Does your PR follow the maintainer guide? Yes

[JackBoosY]

As I know, the compatibility of different versions of python is very poor, so will the upgrade be risky?

[JackBoosY]

Please bump the port version of itk and pybind11. See documentation.

[Hoikas]

Reverting to draft status due to excessive CI failures by ports depending on vcpkg_find_acquire_program(PYTHON3).

[Hoikas]

As I know, the compatibility of different versions of python is very poor, so will the upgrade be risky?

Compatibility between Python versions is generally very good. The different library names (python38 vs python39) are because the ABI does not remain stable between versions. This update should be relatively pain-free. However, 3.10 may be a different story because some deprecated APIs will be removed. That version won't come out for another year though.

Re: CI failures -- this is my fault. I initially pushed a flawed commit for the update to vcpkg_find_acquire_program that has poisoned the Python 3.9.0 tool on the CI machines. If a kind maintainer could delete downloads\tools\python\python-3.9.0-x86 and downloads\tools\python\python-3.9.0-x64, then re-run CI, it should succeed.

I think I have fixed all the issues. I'll keep an eye out for the CI re-run after the tool issue gets fixed. And, of course, please let me know if you see any more problems!

[JackBoosY]

Wow, more regressions. Such as:

[1/4] cmd.exe /C "cd /D D:\buildtrees\glad\src\2f4fb681b3-78068df8f5.clean && D:\downloads\tools\python\python-3.9.0-x64\python.exe -m glad --profile=compatibility --out-path=D:/buildtrees/glad/x64-windows-dbg --api= --generator=c-debug --extensions= --spec=gl --reproducible"
FAILED: src/glad.c include/glad/glad.h 
cmd.exe /C "cd /D D:\buildtrees\glad\src\2f4fb681b3-78068df8f5.clean && D:\downloads\tools\python\python-3.9.0-x64\python.exe -m glad --profile=compatibility --out-path=D:/buildtrees/glad/x64-windows-dbg --api= --generator=c-debug --extensions= --spec=gl --reproducible"
D:\downloads\tools\python\python-3.9.0-x64\python.exe: No module named glad
ninja: build stopped: subcommand failed.

[Hoikas]

@JackBoosY Those regressions are due to the poisoned python-3.9.0-x86 and python-3.9.0-x64 tools on the CI machines. I had an error in the vcpkg_find_acquire_program commit that prevents python from importing 3rd party modules. If someone with access to the machines could delete that and rerun CI, they will be fixed.

[JackBoosY]

@BillyONeal can you please take a look?

[Hoikas]

Sorry for the inconvenience 😢

[BillyONeal]

Unfortunately in the general case we don't have the ability to, like, "run some command on all the build nodes" -- nobody, not even we, can log in to those boxes. They are heavily firewalled because they run arbitrary code submitted in these github PRs and we didn't want someone to be able to write a PR that turns our build lab into a spam fleet, for example.

On the plus side though, the machines are regularly "nuked from orbit" and the VMs are automatically deleted when we don't get lots of incoming build load. I observe that the failures above occurred on BUILD0003H3, BUILD0003H9, and BUILD0003H4, but all the VMs currently alive are BUILD003I? or BUILD003J?, so all the machines on which the failure occurred should long since have been deleted.

[Hoikas]

Great, so a rebase and push should get us going. Thanks for the clarification!

[Hoikas]

CI failures will be fixed by #14571.

[JackBoosY]

Depends on #14579.

[Hoikas]

The remaining CI failures appear to be false-negatives related to downloads.

[strega-nil]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Hoikas]

The failure on arm64_windows is unrelated to this port, which does not function on arm, nor does it consume the PYTHON3 tool.

[strega-nil]

Thanks @Hoikas :)

[linquize]

However, python 3.9 does not support win 7, causing some packages fail to build

[Hoikas]

Hmm... I missed that in the release notes, but sure enough, Python 3.9 dropped support for Windows 7. Windows 7 reached end-of-life on January 14, 2020, so to me, that's a non-issue, especially when the Python release notes are explicitly guaranteeing Big Sur (recent, supported OS) support in 3.9.0. I personally have a dim view of supporting in any way an OS that receives no security patches.

Tangentially related, but it is important for the PYTHON3 tool and python3 port to have the same major.minor version level so that ports providing python extensions will build correctly. So, if we wanted to revert to 3.8, we'd want to back out all but ce20ea6. Then pray that all of the Big Sur changes have been backported to Python 3.8.

[Borgleader]

If Windows 7 is no longer supported by vcpkg, then the Readme needs an update because it still states "Windows 7 or newer" which is evidently no longer true.

I just tried build skia and ran into some issues with Python (even after installing Python 3.8 hoping it would use that instead of the one from the downloads folder). I was confused since it was working months ago the last time I used it and Win 7 still seemed supported (see Readme). Had to dig through the cmake file to realize it explicitly requests 3.9.

This machine is on it's last legs so I amended the file locally to point back to 3.8.3, hopefully I'll be able to keep going with this until my new PC comes along.

[BillyONeal]

@Borgleader vcpkg supports Windows 7. Not every library in vcpkg supports Windows 7. If Python dropped support for it that's largely outside of our control.

[Borgleader]

@BillyONeal I undid the Python upgrade locally and I was able to build Skia without issues (I have been able to draw some text and some shapes based on their SDL2 demo code). So while some libraries may not support Win7 due to Python 3.9 requirement, it doesn't seem to be the case for this one.

(the file i modified to get it to work is: vcpkg_find_acquire_program.cmake)

[linquize]

For those Win 7 users, edit vcpkg_find_acquire_program.cmake to downgrade python to 3.8

[Hoikas]

For those Win 7 users, edit vcpkg_find_acquire_program.cmake to downgrade python to 3.8

Be aware that if you do this without reverting the python3 port itself, you may introduce non-determinant behavior to ports that consume python3.

[hunt978]

we need a versioning design in vcpkg badly !!!

[JackBoosY]

@hunt978 #15172 work in progress now.