Enable building x86_64, arm64 and universal2 wheels on supported macOS Python versions

[dataflake]

This change applies the same environment flags used by cibuildwheel. Specifically, version 10.9 is explicitly selected when building x86_64 (and universal2) wheels. Only arm64 wheels select version 11.0. See here for details:

https://github.com/pypa/cibuildwheel/blob/2e6cbe2435ef410156319c98625002ed76e08785/cibuildwheel/macos.py#L217-L232

I have tried it on AccessControl master and the wheels get built just fine, see https://github.com/zopefoundation/AccessControl/actions/runs/3445282682/jobs/5748817908

You don't happen to know how to get the equivalent of "python version > 3.10" into a GHA expression, do you? That list of specific versions as condition irks me.

[dataflake]

The first cut still failed during BTrees build on macOS/Python 3.11 as setuptools still got the persistent wheel a second time.

I have added a new flag to specify additional packages to install during the Install Build Dependencies step. Installing the dependency (in the case of BTrees it's persistent) at that point will make sure that setuptools knows the egg is already there. It will use that pre-installed egg both for the setup_requires and the install_requires phase of the setup procedure.

[dataflake]

I have now added building x86_64 and arm64 wheels on macOS with Python versions >= 3.8. Python 3.8 is the first version that supports Apple's M-class chips.

arm64 wheels cannot be tested on the GHA runners. So only the x86_64 wheel gets the default name XXX-${{ runner.os }}-${{ matrix.python-version }}.whl and is then picked up by the test step. The arm64 wheel is uploaded as separate artifact so it can be downloaded when desired.

[dataflake]

The first test run of this new configuration is with the BTrees package, see https://github.com/zopefoundation/BTrees/actions/runs/3450155539.

[dataflake]

I will merge this tomorrow morning if there's no feedback. I'm traveling later this week and would like to get this done.

[mauritsvanrees]

The direction seems good, though I can't follow all the details.

I wonder if it would help for relevant packages to use PEP 517 to specify the build system, and then use the build tool to create sdist and wheels. But I did not try any of this, I have only heard of it.

[icemac]

Thank you for putting so much effort into this issue.

[icemac]

This looks okay as this list will shrink when we stop supporting Python < 3.6. (Maybe we could even think about dropping Python 3.6 support.)

[dataflake]

I wanted to make sure it's a list that doesn't need growing as new Python versions are releases. I'm still unhappy that there's apparently no way to safely express something like matrix.python-version < '3.8', something that understands versioning.

[icemac]

Maybe the comment could contain a hint about the --pre flag.
We added it to build AccessControl against Python 3.11beta-X because a dev version of RestrictedPython was needed as its released version was not even installable on Python 3.11.

[dataflake]

Good idea, comment added.

[dataflake]

I wonder if it would help for relevant packages to use PEP 517 to specify the build system, and then use the build tool to create sdist and wheels. But I did not try any of this, I have only heard of it.

I think anything is better than the current situation where we have a mix of packages that either contain copies of header files from other packages (which may drift apart) or that use convoluted code to install requirements and then locate the path to the header files inside the packages. But that's for another day...