Revert #3358 which added libtool versioning to macOS libraries

[nirbheek]

(copying my comment from the PR to this issue)

After discussing with @ePirat it looks like we merged this PR incorrectly. -current_version is not used for any decisions by the linker (so we can set it to be the project version), and -compatibility_version is meant to be set by the developer to signal backwards-compatibility, so that older apps can load newer libraries if the compatibility version matches.

This means that at best, we can set it to soversion. The versioning in this PR is not recommended by Apple, but is what is used by libtool, and we do not use libtool versioning anywhere else in Meson.

It is going to be very confusing if we version libraries on Linux in one way and on macOS using another way.

We should revert this PR.

See:
https://cmake.org/Bug/view.php?id=4383#c9569
https://developer.apple.com/library/content/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/CreatingDynamicLibraries.html

[nirbheek]

CCing @tschoonj @ryandesign

[nirbheek]

I would suggest that we add compatibility_version: as a kwarg to shared_library(), and if it's not specified, it should default to soversion:.

Either that, or we start using libtool versioning consistently everywhere in Meson, which is not something I would recommend.

[tschoonj]

I don't see the confusion really: my patch emulates libtool behaviour on macOS, which is exactly what I wanted it to do. I was always very clear about this.

Just so you know: my patch works fine as it has been applied successfully (in fact the full version from #2577, including the install_name/rpath fix ) in Homebrew for quite some time leading to exactly zero problems.

I also fail to understand why you say that Apple does not recommend the libtool style versioning, even though their own example (from your link) uses it.

Even though the linker does not use -current_version, I think it useful to set it anyway as it can be obtained by the library programmatically by the library to determine its version (from man ld).

[nirbheek]

I also fail to understand why you say that Apple does not recommend the libtool style versioning, even though their own example (from your link) uses it.

Where does it say that? The only reference to libtool I could find was Apple saying that you can use it to generate libraries for you.

As far as I could see, the examples use 1.0 and 1.1 to demonstrate how -compatibility_version is supposed to be used.

I don't see the confusion really: my patch emulates libtool behaviour on macOS, which is exactly what I wanted it to do. I was always very clear about this.

The issue is that we purposely do not use libtool versioning in Meson elsewhere, so it's a matter of consistency.

I can see how this would cause issues in Homebrew because the ABI changes as soon as a project moves to Meson, but as I understand it, that would be solved by doing a rebuild of everything that uses that project? At least that's how Linux distros handle ABI breakage.

Alternatively, you can patch the project to set compatibility_version:, if we implement that as I suggested.

[ePirat]

Apples reference to libtool is very likely about Apples libtool not GNU libtool, I think.

[ePirat]

Note that this issue was originally brought up in the initial PR by @ryandesign, I just pointed out that this was never really addressed before merging the PR.

[ilovezfs]

Apple sets compatibility version and current version on everything in /usr/lib:
https://gist.github.com/ilovezfs/074bb392ae56463ad0a08170d1c11db1

[tschoonj]

I also fail to understand why you say that Apple does not recommend the libtool style versioning, even though their own example (from your link) uses it.
Where does it say that? The only reference to libtool I could find was Apple saying that you can use it to generate libraries for you.

As far as I could see, the examples use 1.0 and 1.1 to demonstrate how -compatibility_version is supposed to be used.

Indeed, very similar to what my patch does. Before my patch there was no use of -compatilibity_version or -current_version in meson at all. Now they are set to proper values.

I can see how this would cause issues in Homebrew because the ABI changes as soon as a project moves to Meson, but as I understand it, that would be solved by doing a rebuild of everything that uses that project? At least that's how Linux distros handle ABI breakage.

Rebuilding is something that we avoid whenever possible as it is very time-consuming, and causes trouble for users who have built their own libraries against an old version of a library: when updated with the package manager, their own own library will refuse to load at run time.

Alternatively, you can patch the project to set compatibility_version:, if we implement that as I suggested.

[nirbheek]

Indeed, very similar to what my patch does. Before my patch there was no use of -compatilibity_version or -current_version in meson at all. Now they are set to proper values.

I am not disputing whether we should set -compatibility_version or not. The question is what do we set it to. After your patch, they are set to follow libtool's versioning scheme. I do not think @jpakkane realized that when we merged your PR since in the past he has been very vocal about not using libtool versioning (for linux shared library naming, for instance).

Apple sets compatibility version and current version on everything in /usr/lib:
https://gist.github.com/ilovezfs/074bb392ae56463ad0a08170d1c11db1

Thanks for that link. It seems Apple does not use libtool versioning. From what I can tell, they set it to x.0.0.

Would be great to figure out what XCode does for this.

[ryandesign]

I am not disputing whether we should set -compatibility_version or not. The question is what do we set it to. After your patch, they are set to follow libtool's versioning scheme.

As I mentioned in the previous PR, I am not convinced that they are. GNU libtool is able to compute the library's major version, minor current version and minor compatibility version because the library author has to follow very specific rules when specifying the -version-info parameter, using current:revision:age. current:revision:age is not a version number. Unless I misunderstand, a developer making a library with meson does not specify the library's libtool-style current:revision:age anywhere, so libtool-style algorithms that compute versions based on them can't be used.

[tschoonj]

In my initial exchange with @jpakkane (as well as the PR title...) in #2577 I made it abundantly clear that I was trying to emulate the libtool versioning scheme on macOS. I appreciate that libtool is problematic in many ways but this particular thing it does well. This did not appear to be a problem at the time.

[ryandesign]

Would be great to figure out what XCode does for this.

I don't think that will be instructive. Since Xcode is a Mac-specific IDE, it merely lets the developer specify the library's major version, minor current version and minor compatibility version directly. The fact that these concepts don't map one-to-one to how other operating systems version their libraries is not Xcode's concern.

[tschoonj]

From what I understand, GNOME projects that migrated to meson are now putting hacks into their meson.build scripts to ensure compatibility with the old libtool versioning scheme. This was already working fine with Linux builds, didn't break backwards compatibility, and my patch ensured that this would also be true for macOS.

Some projects are manually adding -compatibility_version and -current_version flags to the meson.build ldflags: sometimes properly, sometimes not...

From json-glib:

https://github.com/GNOME/json-glib/blob/98a1d1f3860acb77f800f93c8d2cb891d7238f34/meson.build#L10-L27

# Versioning
json_version = meson.project_version()
version_arr = json_version.split('.')
json_version_major = version_arr[0].to_int()
json_version_minor = version_arr[1].to_int()
json_version_micro = version_arr[2].to_int()


json_interface_age = json_version_minor.is_even() ? json_version_micro : 0
json_binary_age = 100 * json_version_minor + json_version_micro
json_api_version = '1.0'


json_api_name = '@0@-@1@'.format(meson.project_name(), json_api_version)
json_gettext_domain = json_api_name


# Maintain compatibility with the old soname versioning
soversion = 0
libversion = '@0@.@1@.@2@'.format(soversion, json_binary_age - json_interface_age, json_interface_age)

and

https://github.com/GNOME/json-glib/blob/98a1d1f3860acb77f800f93c8d2cb891d7238f34/meson.build#L142-L148

# Maintain compatibility with autotools
if host_system == 'darwin'
  common_ldflags += [
    '-compatibility_version 1',
    '-current_version @0@.@1@'.format(json_binary_age - json_interface_age, json_interface_age),
  ]
endif

[ryandesign]

I can see how this would cause issues in Homebrew because the ABI changes as soon as a project moves to Meson, but as I understand it, that would be solved by doing a rebuild of everything that uses that project? At least that's how Linux distros handle ABI breakage.

Certainly, if the library changes in an incompatible way (e.g. a public function is removed), increase its major version number so that all consumers must rebuild to link with the new library. But if the library has not changed incompatibly, don't change the major version! This and other aspects of sane library version number curation are the responsibility of each developer who makes a library.

Unfortunately, many developers don't realize that they have this responsibility, or don't realize that switching build systems will have an impact on it. This happened recently with the release of libebml 1.3.6 and libmatroska 1.4.9 in which the developers switched from autotools with libtool to cmake, which had the unintended side effect of decreasing the library's minor version numbers on macOS thus making the new library incompatible with all software that had been linked with the previous version. The developers didn't realize that libtool uses a different library version numbering scheme on macOS than it does on Linux, while cmake doesn't have that difference.

I'm not sure how far a build system can go mitigate this problem. It's not necessarily meson's problem that libtool uses the library versioning scheme it does. If meson adopted libtool's versioning scheme, with all of its platform differences, that would help projects coming to meson from libtool, but wouldn't help those coming from cmake or other build systems. Perhaps it would be helpful to document information about this problem and how to avoid it, if that hasn't already been done.

[jpakkane]

The problem of Libtool's versioning scheme is that the we have to fully support it and all its bugs and I'm fairly sure it is not properly documented anywhere. Also libtool as a project is as good as dead. If we ever need to support a platform that libtool does not, then we are again in trouble. And in fact I would not be that surprised if Apple removed their own libtool altogether in some future release of macOS. They have done it before with e.g. OpenSSL.

If we could have exact, specific and unambiguous data that does "the right thing" or a fairly good approximation thereof on all platforms, then that would probably the better thing to do.

[ebassi]

From what I understand, GNOME projects that migrated to meson are now putting hacks into their meson.build scripts to ensure compatibility with the old libtool versioning scheme.

Just to be abundantly clear (since you linked to my project): the only reason those hacks were added was because you opened a bug about it asking for backward compatibility with libtool in homebrew. I'd be happy to:

• remove the hack today
• not care at all about libtool back compatibility

[tschoonj]

@ebassi That's only true for the second hack involving -compatibility_version. I cannot claim any credit regarding the first one.

[tschoonj]

@jpakkane Apple's libtool is not used by Autotools. The libtool that gets used during a build is actually a script that is generated when running configure, and which does not make use of Apple's libtool.

[nirbheek]

As per discussion on IRC with @jpakkane, we will revert the original patch for 0.46.1, and add new API in 0.47 which will allow projects to set -compatibility_version to a value that they desire. Since Homebrew is already patching Meson, they can reinstate the full patch from #2577 for this release too.

<nirbheek> So for 0.46.1 we revert that patch?
<nirbheek> And in 0.47 we can add new API
<nirbheek> Homebrew can just continue to carry their old patch for 0.46.1 till we come to a decision
<jpakkane> Seems like a reasonable approach.

I will work on a mechanism that allows projects to easily set the version/soversion/compatibility_version/current_version/etc to more accurately match libtool's versioning if they have releases to maintain ABI compatibility with. Will open a new bug about it.

[tschoonj]

Ok, fair enough. Hope this will get resolved in 0.47 then.

Homebrew currently has two meson formulas: meson and meson-internal. The former is your release, clean, while the latter is currently at 0.45.1 (not 0.46.1), with the patch from #2577.

We use meson-internal to build our GNOME formulas, as it provides libtool compatible versioning and produces working gobject-introspection bindings.

Users that just want to build their own projects with meson are expected to use the regular meson.

[nirbheek]

We use meson-internal to build our GNOME formulas, as it provides libtool compatible versioning and produces working gobject-introspection bindings.

Users that just want to build their own projects with meson are expected to use the regular meson.

I'm happy to hear that, I was concerned that people that are using meson in their workflow might be affected/confused by the patch you have been carrying. Glad that you already thought of it!