MinGW32 cannot build Python wheel packages

[bcoconni]

When building JSBSim with -DBUILD_PYTHON_MODULE=ON, the library is built with MinGW32's gcc compiler but, for some reason, the Python module is built with MSVC which triggers a complete new compilation of the library. As a result, the library is compiled twice.

That's a shame because I would prefer building the Python modules for Windows with MinGW32 rather than MSVC. That would avoid shipping the DLL msvcp140.dll and its questionable license with our Python Windows package.

[seanmcleod]

So to refresh my memory I took a look at #73 which is where the msvcp140.dll first cropped up.

I didn't see us discussing the option to link statically with the C/C++ library at the time as one option as opposed to distributing msvcp140.dll.

In terms of a questionable license I'm not sure what the potential issue is. This link from Microsoft https://docs.microsoft.com/en-us/visualstudio/productinfo/2015-redistribution-vs mentions:

you may copy and distribute with your program any of the files within the followng folder and its subfolders except as noted below. You may not modify these files.

C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist

I just double-checked and and msvcp140.dll does live in that directory tree.

Now while looking around for some info on Python modules and msvcp140.dll I came across this very detailed explanation from one of the Python developers.

As I’m largely responsible for the compiler change (with the full support of the CPython developers, of course)

https://stevedower.id.au/blog/building-for-python-3-5

And the recommendation at the end of it is to link statically against the C/C++ runtimes.

However he mentions that distutils from python 3.5 onwards has been updated to build python modules using static linking. So does that mean we're using a pre-python 3.5 version of distutils when we build our python module?

Lastly right at the end of the page there is mention of MinGW and compatibility issues. Although the post was written in Aug 2015 so maybe they've been addressed since then?

[bcoconni]

Thanks for the detailed answer @seanmcleod, that was very helpful.

I didn't see us discussing the option to link statically with the C/C++ library at the time as one option as opposed to distributing msvcp140.dll.

Indeed, we haven't discussed such an option after we resolved the issue #73.

Since it's quite a looong answer that I have written below, here is the executive summary aka TL;DR:

• It seems there is no license issues about shipping msvcp140.dll with our Python packages because our license is the LGPL 2.1 and because our packages are distributed by GitHub who (hopefully) own a valid copy of MSVC. It would have been a different story if JSBSim was licensed under the GPL.
• Python build tools distutils have dropped support for MinGW32 (and cygwin) since Python 3.4 (which is now officially obsolete). As an obvious consequence, we cannot build our Python wheel packages with MinGW32.
• The only viable option seems to be dynamically linking JSBSim with msvcp140.dll and ship it with our Python packages.

Now here is the detailed answer. Brace yourselves:

License

In terms of a questionable license I'm not sure what the potential issue is. This link from Microsoft https://docs.microsoft.com/en-us/visualstudio/productinfo/2015-redistribution-vs mentions:

you may copy and distribute with your program any of the files within the followng folder and its subfolders except as noted below. You may not modify these files.

C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist

Yes, I am aware of that statement however your quotation is incomplete (bold-facing is mine):

Subject to the License Terms for the software, you may copy and distribute with your program any of the files within the following folder and its subfolders except as noted below.

Which license are they referring to ? For instance the top paragraph of Distributable Code Files for Visual Studio 2017 say (bold-facing is mine again):

The following section is the "REDIST list" that is referenced in the "Distributable Code" section of the Microsoft Software License Terms for Visual Studio Enterprise 2017, Visual Studio Professional 2017, Visual Studio Community 2017 ("the software"). If you have a validly licensed copy of such software, you may copy and distribute with your program the unmodified form of the files listed below, subject to the License Terms for the software.

I infer from that statement that you need to own a license of Visual Studio to be granted the right to copy and distribute the aforementioned files. In the case of our CI workflow, GitHub own licenses of Visual Studio (at least, I hope they do 😉) and our packages are distributed through their site so I can only guess that we are covered by their license.

In addition, a lot of open source projects (including Python) are happily distributing some files from the redist folder with their packages so it seems the redistribution of msvcp140.dll and the likes is not frowned upon by Microsoft.

For a bit of context, I started being involved in open source in the early 2000 which was the time when Steve Ballmer was Microsoft CEO. Remember that guy who said that open source was a "communist cancer" ? He did all he could to shoot down open source projects with software patents, undisclosed APIs and purposely standards violations. Although his successor, Satya Nadella, is unquestionably having a friendly attitude towards open source (and this is why I did not flee from GitHub when Microsoft purchased them), it remains that, as far as I am concerned, Steve Ballmer's era left some scars. Hence my uneasiness.

In addition to that, I have come across this answer from the GNU foundation in a FAQ about the GPL license

I'm writing a Windows application with Microsoft Visual C++ (or Visual Basic) and I will be releasing it under the GPL. Is dynamically linking my program with the Visual C++ (or Visual Basic) runtime library permitted under the GPL?

[...] You may not distribute these libraries in compiled DLL form with the program. To prevent unscrupulous distributors from trying to use the System Library exception as a loophole, the GPL says that libraries can only qualify as System Libraries as long as they're not distributed with the program itself.

Fortunately, JSBSim is licensed under the LGPL 2.1 which allows distribution of JSBSim with msvcp140.dll under section 7 of the LGPL 2.1

7. You may place library facilities that are a work based on the Library side-by-side in a single library together with other library facilities not covered by this License, and distribute such a combined library, provided that the separate distribution of the work based on the Library and of the other library facilities is otherwise permitted, and provided that you do these two things:

• a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities. This must be distributed under the terms of the Sections above.
• b) Give prominent notice with the combined library of the fact that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work.

However, the particular case of the GPL license that I mentioned above shows that licenses are, at least to me, muddy waters.

Compilation with MinGW32

I have finally fixed the problem that triggered the compilation of the Python package with MSVC even when mingw32-make was called. The result is that, with MinGW32's gcc compiler, the Python package compilation fails miserably with the error message ValueError: Unknown MS Compiler version 1916.

As a matter of fact, the Python's distutils package gives up compilation when using MinGW32 along with a version of Python that has been compiled with MSVC more recent than VS2010 (lines 61 to 84 in cpython/Lib/distutils/cygwinccompiler.py)

def get_msvcr():
    """Include the appropriate MSVC runtime library if Python was built
    with MSVC 7.0 or later.
    """
    msc_pos = sys.version.find('MSC v.')
    if msc_pos != -1:
        msc_ver = sys.version[msc_pos+6:msc_pos+10]
        if msc_ver == '1300':
            # MSVC 7.0
            return ['msvcr70']
        elif msc_ver == '1310':
            # MSVC 7.1
            return ['msvcr71']
        elif msc_ver == '1400':
            # VS2005 / MSVC 8.0
            return ['msvcr80']
        elif msc_ver == '1500':
            # VS2008 / MSVC 9.0
            return ['msvcr90']
        elif msc_ver == '1600':
            # VS2010 / MSVC 10.0
            return ['msvcr100']
        else:
            raise ValueError("Unknown MS Compiler version %s " % msc_ver)

Python Wiki tells no different story about Windows compilers:

Visual C++	CPython
14.X	3.5, 3.6, 3.7, 3.8
10.0	3.3, 3.4
9.0	2.6, 2.7, 3.0, 3.1, 3.2

And the section about MinGW in the same Wiki article clearly states that: "MinGW is an alternative C/C++ compiler that works with all Python versions up to 3.4."

So much for compiling our Python package with MinGW32 !

Anyway, I have pushed the code to compile the Python package with MinGW32 to JSBSim, in case the Python developers change their mind in the future. For now, the compilation of our Python package is disabled for MinGW32 by setting BUILD_PYTHON_MODULE to OFF.

Shipping msvcp140.dll vs static compilation

So the only option remaining is dealing with MSVC runtime libraires.

Thank you @seanmcleod for bringing to my attention the blog post from Steve Dower. It was an enlightening reading since Python developers are obviously facing the same problem than us with the only difference that their piece of turd is named vcruntime140.dll.

Static compilation delicacies

One way to avoid shipping msvcp140.dll with our Python package would be to statically link it with JSBSim. That's the solution that is pushed by Steve Dower in his original post but which he mitigated in a later post.

In addition to being quite involved (static compilation but dynamic linking), his solution can trigger a problem due to the hard limit set by Windows on the number of fiber local storages (whatever they are, their limit seems to be 120-ish per process). As a result Python developers designed a process where Python packages would be statically linked against vcruntime140.dll unless distutils is told otherwise (i.e. if you are tinkering with the obscure flags DISTUTILS_USE_SDK and PY_VCRUNTIME_REDIST).

When brown matter hit the fan

This design proved to be robust enough until late 2019 (Python bug report 38597), when that decision blew up to everyone's face after people have started reporting problems when using more than 120 packages/extensions (rings a bell ?) at the same time. Given that some packages depend on packages that depend on packages that depend on packages, etc. the occurrence of such an event might not be as low as was initially expected. To address this issue, Steve Dowler had no choice but to commit a change to distutils which is the complete opposite option to what he had advocated in his 2015 posts:

distutils will no longer statically link vcruntime140.dll when a redistributable version is unavailable. All future releases of CPython will include a copy of this DLL to ensure distributed extensions can continue to load.

In other words, the statically linked option has been dropped and the required runtime libraries will be shipped with Python.

To be honest, I raise my hat to Steve Dowler who has been very creative trying to address a problem which is really complicated but the odds were against him.

Even Microsoft discourage static linking

In their documentation, Microsoft advise against static linking to their runtime libraries because it can hinder security updates:

[...] For example, consider an application that's statically linked to a library that's been updated with security enhancements—the application cannot benefit unless it is recompiled and redeployed.

Caveats of shipping msvcp140.dll

Beyond the license issue that we already discussed, shipping msvcp140.dll comes at a price:

• The package size is bigger but I really don't think it's a problem given that our Python packages are quite small (approx. 1,5 MB).
• Microsoft do not recommend to ship their runtime library for the same reasons than they are advising against static linking:

[...] You can use local deployment to install the required Visual C++ DLLs in the directory of a particular user's application without affecting other users or requiring administrator rights. Because this can create serviceability issues, we do not recommend local deployment of Visual C++ redistributable DLLs.

Instead of shipping their runtime libraries, Microsoft are therefore suggesting to use their redistributable packages (VCRedist_x86.exe, VCRedist_x64.exe, or VCRedist_arm.exe) which install the libraries system-wide and register them as candidates for security/enhancement updates by the OS.

However, as Steve Dower rightly stated, this would basically disqualify JSBSim wheels from using PyPI, since we would also need to get our users to install the VCRedist version that matches the compiler that was used to build the packages. And, currently, there is no way to check or enforce this through tools like pip.

Consequences for JSBSim

So obviously, static linking is no longer an option since Python developers have stepped away from it and even Microsoft does recommend against that.

The runtime library msvcp140.dll then needs to be brought to the user system either shipped along with JSBSim wheels or distributed via the installation of one of Microsoft VCRedist.exe package. Unfortunately, the latter option would imply that JSBSim Python wheels could not be distributed via pip and that leaves us with only one remaining option: shipping msvcp140.dll with JSBSim Python wheels for Windows.

What about Linux and MacOSX ?

Still there ? Not asleep yet ? Congratulations 🎉

Indeed, the other operating systems have their own variation of the "DLL hell".

Linux

Regarding Linux, the process albeit as involved as for Windows seems to have settled with the following PEPs:

• PEP 513 manylinux1,
• PEP 571 manylinux2010,
• PEP 599 manylinux2014.

Basically the idea is to build Python packages with a very old Linux distribution (CentOS 5.11 released in 2007 and obsolete since 2017) and to ship the required system libraries with the packages (no license issues involved there since Linux system libraries are open source). Other PEPs (571 and 599) allow more recent versions of CentOS to be used but at the expanse of the number of supported Linux distributions (the higher the number after "manylinux", the lower the number of supported Linux distributions).

Incidentally, it seems that JSBSim does not use recent features of Linux system libraries so our wheels are rated manylinux1 i.e. with the broader compatibility across Linux distributions.

Although the management of system libraries is automated by auditwheels, the process of generating wheel packages for Linux is not that simple since we have to compile and build JSBSim in a docker container that emulates that old Linux distribution that the PEPs request, a task driven by the script jsbsim/python/build-wheels.sh.

Mac OSX

I have yet no idea about how the dependencies to system libraries are managed on MacOSX. At the moment, our CI workflow manages to build and test successfully Python packages for MacOSX. Yet, I don't know how good they are when installed on the average Macbook.

The strategy here is to basically wait until someone posts an issue on GitHub reporting that the wheel package does not work on his/her Macbook.

Feedback

What do you all think ? Any thought on these topics ?

[seanmcleod]

Hmm, well your answer was 10x more detailed than my answer :wink

I'd suggest maintaining our status quo of shipping msvcp140.dll as part of the JSBSim wheel.

[bcoconni]

Hmm, well your answer was 10x more detailed than my answer 😉

😄 I was not trying to compete with your answer but rather building on the foundations it has laid. When I said the links you have provided were enlightening, I really meant it 👍. I spent a whole afternoon gathering additional information and making up my mind on that topic, so I felt it needed to be documented somewhere for future reference. Hence that long answer.

I'd suggest maintaining our status quo of shipping msvcp140.dll as part of the JSBSim wheel.

Good, so I consider the issue closed.