Use max compatible Python interpreter rather than minimum #7048
Conversation
On macOS, if the interpreter is not available when running tests, then it throws an OSError for file not found. This is currently not being caught as intended.
|
This is the opposite of pex behavior and I can only imagine this makes confusing interpreter selection choices that much more confusing. Is this really needed? It just supports our internal dev efforts afaict.
This bit sounds wrong / broken and is what should be fixed imo. |
|
Closing in favor of simply using As discussed with John over Slack, this may lead to a compatibility issue when trying to run the Antlr tests in CI. If this does become a problem and we can't resolve it otherwise, we'll reconsider this solution. |
There is a matrix of what interpreter we use for under-the-hood and what we use for subprocesses. Presumably, when calling `./pants3`, you would expect to also run your tests and subprocesses with Py3. This change is not only nice-to-have but also required because the native engine does not work when bootstrapped with Python 3 and running a Python 2 subprocess *if* that subprocess uses `native_engine.so`, as several of our tests do. This may result in conflicts when trying to run Py2-only targets, e.g. python_antlr_library.py. If this is the case, we can resort to pantsbuild#7048 as an alternative solution.
Problem
When running
./pants3, we would expect to run subprocesses like Pytest using Python 3 as well. Instead, we resort to using Python 2—as it's the minimum acceptable—unless--python-setup-interpreter-constraintsis explicitly set to require Python 3.One workaround to this issue would be setting
PANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTSto only allow Python 3, but this would mean that./pants3could no longer be used to run targets that require Python 2 like Antlr.Further, we in general want to write Python 3-first code. Defaulting to the minimum interpreter possible does not coincide with this objective.
How to get to prior behavior
Users may go back to using the minimum interpreter through several ways:
--python-setup-interpreter-constraintsflagPANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTSpants.iniandcompatibilityBUILDfiles andcompatibilityImplications
./pants3script #6959, Pants and CI will default to using Python 3 for subprocesses instead of Python 2—even when using Python 2 under-the-hood. This is intentional. We can use the compatibility mechanisms to force Python 2 (e.g. CI will usePANTS_PYTHON_SETUP_INTERPRETER_CONSTRAINTSto get our Py2 shards to use Python 2).brew upgradewhen 3.8 comes out, and used our internal compatibility setting of>=2.7,<3 or >=3.6,<4and have issues like not usingcollections.abc, then they will get aSyntaxError. Keep in mind this only happens one a year, though, when a new version is released. Further, it is easy to constrain their compatibility to no longer allow 3.8.