Include libusb in the Mac App bundle

[TheaMorin]

Summary

The Stenograph USB plugin requires translation of third-party library libusb to work on Mac. On Windows, the user uses the manufacturer's driver, on Linux a compatible backend is usually installed.

Implementation Notes

I've been playing with trying to get this to work on my local machine. It seems it's not possible to point the python executable in the app bundle to libusb using the preferred method, because pyusb uses find_library() which just checks certain directories.

It might be possible to add our app's bundle to the checked directories by setting $DYLD_LIBRARY_PATH to the app's Frameworks folder with the libusb dylib in it.

I also found that the dylib has some static links which might be good to set after moving the libusb file: install_name_tool -id "@executable_path/libusb.dylib" Plover.app/Contents/Frameworks/libusb.dylib (though it's not exactly clear to me what the benefit is, here)

The other option if we can't control find_library with environment variables is to provide a function to pyusb that knows where to look. The disadvantage here is that the code would move into the plugin.

[TheaMorin]

I modified the launch script to have:

DYLD_LIBRARY_PATH="$DIR/../Frameworks" exec "$DIR/../Frameworks/python3.6" -m plover.main "$@"

with libusb.dylib copied into Frameworks, and it appears to work.

[benoit-pierre]

Regarding static links: doesn't running macholib takes care of that? As for the DYLD_LIBRARY_PATH it looks like the equivalent of LD_LIBRARY_PATH on Linux, if it works, then let's use it.

[TheaMorin]

Macholib fixes the links within the python3.6 executable. I don't believe that it handles direct dylibs that are loaded dynamically at runtime.

…

On Feb 28, 2018 8:29 PM, "Benoit Pierre" ***@***.***> wrote: Regarding static links: doesn't running macholib takes care of that? As for the DYLD_LIBRARY_PATH it looks like the equivalent of LD_LIBRARY_PATH on Linux, if it works, then let's use it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#904 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFkgSj78jxG3krUSIi9sayjG1z-RDPIFks5tZ08DgaJpZM4SWvbJ> .

[benoit-pierre]

I thought it did: https://macholib.readthedocs.io/en/latest/scripts.html#python-m-macholib-standalone

[TheaMorin]

Okay, that does look promising, just copy the libusb Dylib, run macholib, and it copies it over and fixes the references? I will be trying that out.

…

On Feb 28, 2018 10:17 PM, "Benoit Pierre" ***@***.***> wrote: I thought it did: https://macholib.readthedocs.io/en/latest/scripts.html# python-m-macholib-standalone — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#904 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFkgSjVWMyIgEqpRz5ogefXIsw9jelgSks5tZ2g5gaJpZM4SWvbJ> .

[benoit-pierre]

Yep, exactly.

[TheaMorin]

Okay, so I've got my proof-of-concept going

diff --git a/osx/app_resources/plover_launcher.sh b/osx/app_resources/plover_launcher.sh
index 91fba813..5be904aa 100644
--- a/osx/app_resources/plover_launcher.sh
+++ b/osx/app_resources/plover_launcher.sh
@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
 set -e
 DIR=$(dirname "$0")
-exec "$DIR/../Frameworks/pythonexecutable" -m plover.main "$@"
+DYLD_LIBRARY_PATH="$DIR/../Frameworks" exec "$DIR/../Frameworks/pythonexecutable" -m plover.main "$@"
 
diff --git a/osx/make_app.sh b/osx/make_app.sh
index 4b6b85a5..15d2c6d6 100644
--- a/osx/make_app.sh
+++ b/osx/make_app.sh
@@ -45,6 +45,7 @@ mkdir "$app_dir/Contents/Resources"
 cp "$python3_dir/Resources/Python.app/Contents/MacOS/Python" "$target_dir/${target_python}"
 
 # Copy over system dependencies for Python
+cp ~/libusb.dylib $target_dir
 "$python" -m macholib standalone "$app_dir"
 
 # Make site-packages local
diff --git a/requirements_plugins.txt b/requirements_plugins.txt
index b8483502..aa2cc63f 100644
--- a/requirements_plugins.txt
+++ b/requirements_plugins.txt
@@ -7,5 +7,6 @@ wheel==0.30.0
 # plover-treal
 hidapi==0.7.99.post21
 plover-treal==1.0.1
+plover-stenograph-usb
 
 # vim: ft=cfg commentstring=#\ %s list

but I'm not sure how to get the dylib without brew. Isn't using brew not an issue as long as we are okay using the latest version?

[benoit-pierre]

Is that still an issue?

libusb1 support having the library alongside the Python code (in the package directory). There's a wheel for Windows that includes the necessary DLL. No such wheel for macOS, but it seems like just plopping a version of libusb in the right directory might do the trick.

[TheaMorin]

The devs of pyusb told me that I could provide a custom backend-finding function to the "find" function.

What I'm thinking is Plover would add pyusb as a dependency, and we would include the dylib plus export a get_bundled_backend that falls back to the built-in one if run outside of a bundle.

Then plugins could make use of that in case anyone else wants to interface with usb

[TheaMorin]

one problem I'm running into is that we're using the app bundle executable Contents/Frameworks/Python.framework/Versions/Current/Resources/Python.app

I think it would be better to use the bundled executable Contents/Frameworks/Python.framework/Versions/Current/bin/python—that would remove the use of _NSGetExecutablePath right?

[benoit-pierre]

Sorry, I'm lost, do you have a POC I can look at?

[TheaMorin]

I'll gather more details first :) but basically when I'm getting notifications from plyer, they're showing as from Python.app rather than Plover.app. This is since changing to the portable python framework with the GitHub Actions PR.

I think that this is a symptom of the fact that we're running an app bundle with its own PList within an app bundle and that's going to cause us some issues (this should probably be treated as a separate issue from libusb).

[benoit-pierre]

OK, do we use plyer on macOS? I thought it was a Windows only dep.

[TheaMorin]

Yeah, sorry—carelessness on my part—it's the NSUserNotification notifications that are showing up as from python.

[TheaMorin]

If you look at the difference between a relocatable-python build and the way we were doing it previously, it seems like the relocatable-python framework always wants to run python through the Resources/Python.app/Contents/MacOS/Python—which is completely absent from the macholib'd framework.

Currently in plover_launcher.c we're running Frameworks/Python.framework/Versions/Current/bin/python. If I delete Python.app and then launch Plover, I get:

$ ./Plover.app/Contents/MacOS/Plover
python: posix_spawn: /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python: Undefined error: 0

I'm not sure why there's a discrepancy between macholibbing our own frameworks folder and relocatable-python, but it would be nice to figure out how we can not deliver a bundled Python.app

[benoit-pierre]

I see, I don't remember the exact error I had with the old method. I ran into issues both when updating to a new Python, and switching to Github Actions.

[TheaMorin]

Okay, since the libusb issue is separate from the notification, I created #1237

For libusb, in our last discussion you suggested publishing a separate plover-usb package that includes the dylib in the wheel. I think I like that approach better than adding an extra to plover and I'll take a stab at it.

[benoit-pierre]

Closing, as plover_stenograph_usb depending on pyusb_libusb1_backend takes care of the problem.