[Fix] virtualenv python interpreter used

[idantene]

In a recent struggle with clearml-agent, I discovered an odd edge case, leading to this PR.

In my case, there are 4 different Python interpreters on the system - 2.7, 3.6.9, 3.7.5, and 3.8. For Python 3.6.9 and Python 3.8, the virtualenv in question was either most up to date or relatively up to date. For Python 3.7.5 it was extremely outdated.

Then, when a user had requested to use Python 3.7.5, the agent would correctly identify the executable (python3.7) and run python3.7 -m virtualenv <path>. For some reason, the old install of virtualenv consistently used python2 as the underlying executable, and any task would then crash.

Finding the source for this bug required a deeper dive into clearml-agent code, and I believe simple logging of the virtualenv version used would've been useful.
Finally, the use of --python <python interpreter> would've worked just as well, forcing virtualenv to use the specific interpreter (which should be the default, but apparently it was not).

Hence this PR:

1. Add a simple print of the virtualenv version using pythonx.y -m virtualenv --version
2. Force install the correct python interpreter using --python flag

[JDennisJ]

Hi @idantene ,

Can you send a small toy example for reproducing your original issue before the fix?

[idantene]

Hey @JDennisJ!

Unfortunately not really, as it's not code-specific.
You can and mimic the aforementioned setup locally:

1. Have multiple Python installs on the same machine. Make sure Python 2.7 is also available.
2. In one of these, say Python 3.7.5, install an old version of virtualenv (at least pre 20.0.0, but since it was not logged, I'm not really sure which exact version it was). Use python3.7 -m pip install ... to ensure the old virtualenv version is local to that Python interpreter.
3. Ensure other Python versions have the latest virtualenv.
4. Run clearml-agent on one of these Pythons with latest virtualenv.
5. Send a task for execution that specifically asks for Python 3.7.5.

If the "correct" old virtualenv version was installed, then python3.7 -m virtualenv ... will use the Python 2.7 interpreter. There is no indication that an old virtualenv version used, and the error could've anyway been avoided by specifying the python interpreter for virtualenv.

[jkhenning]

@JDennisJ any news on this?

[JDennisJ]

Hi, I think we can merge it :)