Fix MacPorts Issue. Could not open library dlopen

[comomac]

I had issue trying to load the library, which is caused by looking at wrong place for library.

My setup is using MacPorts which may be the cause. I've setup to auto detect the environment (macos) and if the library exist, will use the fullpath of the library.

[jcupitt]

Hello @comomac, thank you for this patch.

I've been trying to read around this issue, and it seems to be something that's causing problems for quite a few people, for example:

ffi/ffi#461

It seems we need to search for our libraries on macOS, especially if SIP is enabled. This is very annoying!

We need to try:

1. Just the plain name. This should work with homebrew, or if the user has built their own libvips and Ruby and set DYLD_SEARCH_PATH.

2. Turn the name into a library name by adding lib at the front and .dylib at the back.

3. Now look in /opt/local/lib for macports.

4. /sw/local/lib for Fink (I think?).

What do you think? We need to add this extra search to ffi_lib somehow.

[comomac]

Did some quick reading.

I didn't have to disable SIP because my environment didn't need it yet (MacPorts). But it does seems like /usr/local is one of the path that Apple SIP is preventing which causes headache for Brew and others.

1. sure
2. I've found out the ffi_lib adds lib and .dylib automatically if using just plain name under MacOS env, so can probably omit since cover by 1. (LInux env automatically add .so etc)
3. sure
4. IIRC Fink uses the same /opt/local/lib path as MacPorts, probably can omit this since cover by 3

I've tried to find out if there is a in-built method (Ruby and FFI) to have an list of library path to search in order like environment PATH variable in shell but for library, but came out empty.

I've thinking that if we could fix path issue, it will solve majority of user's problem (there are probably a lot people experience it but didn't report) when they first tried using the library.

If path issue isn't fixed this round, we will fix it next round and learn more from experience and hopefully find out the best way to solve path search issue. Or maybe FFI project may just come up with path search instead (laugh).

[Nakilon]

@comomac have you tried to recheck your problems on High Sierra?

[ioquatix]

One option to fix this is to compile glib directly into the gem.

I started working on something like that here: https://github.com/ioquatix/vips but for now it only compiles libvips.

[comomac]

Forgot to check message. I'm running Sierra so I don't know if the problem persist in High Sierra.

[jcupitt]

I came across this in the ruby ffi sources:

https://github.com/ffi/ffi/blob/master/lib/ffi/library.rb#L127

                # TODO better library lookup logic
                unless libname.start_with?("/") || FFI::Platform.windows? 
                    path = ['/usr/lib/','/usr/local/lib/'].find do |pth|

Perhaps everyone knows about this. Anyway, it seems that:

• macOS disregards DYLD_LIBRARY_PATH for Apple executables (things in /usr/bin) for security, so you can't make it open libraries in non-standard places that way
• ruby-ffi will automatically check /usr/lib and /usr/local/lib, but nowhere else

The best solution would probably be for ruby-ffi to have its own dyld path on macOS (on Windows there's RUBY_DLL_PATH, I believe). Failing that, ruby-vips probably needs a very ugly magic search hack :-(

[comomac]

@jcupitt Thanks, I was looking for answer on FFI .dylib search path and that line answers it, maybe it is worthwhile to suggest FFI to add /opt/local/lib into search path on that exact line. It will reduce a lot of headache for many other libraries that rely on MacPorts. Would you like to suggest a patch to FFI? I can do I if you are too busy.

[Nakilon]

Just a reminder: my High Sierra doesn't have the issue. You might want to upgrade because old system Ruby (2.0) is with an old OpenSSL that makes it unusable in some scenarios (and maybe this is why brew now downloads Ruby for itself -- just a guess).

[jcupitt]

@comomac sure, PR away! I have loads of work this weekend :( I'm sure they'd have expert insights, even if this change is not quite correct.

@Nakilon my High Sierra machine has this issue, or at least a variant of it. If I use the Apple Ruby and install libvips to a non-standard prefix, I can't get ruby-ffi to open my libvips.dylib without giving the exact path in the dlopen() line.

[Nakilon]

to a non-standard prefix

Oh I missed this point.

[comomac]

close.

the path is added to ffi/ffi#638 master branch

[jcupitt]

Thanks @comomac ! Nice work.

[ioquatix]

Thanks for this!