Building on M1/M2 Macs

[achingbrain]

Would it be possible to ship a precompiled binary for arm64 on Mac OS X along with the x64 versions? Installing this module takes a very long time on Apple silicon and requires lots of extra tools to be installed.

It seems github actions do not support Apple M1/M2 processors so some sort of cross compilation step would be necessary?

[martenrichter]

Currently not, since GitHub action does not support it.
(Required would be cross-compilation or GitHub runner hosted somewhere else).
Anyway, cross-compilation is not an easy task and it is not on my agenda.
But if someone managed to implement it, I would guide the process and after review includes a PR to the CI/CD pipeline.

[achingbrain]

Do you know if you can build universal binaries for x64/arm64 with cmake-js? I'm not really familiar with it.

The leveldown project happily builds universal binaries on GitHub CI but I think they use a different build system.

[achingbrain]

Actually, I say that, but the node-datachannel module uses cmake-js and builds m1 binaries so it looks like there's some config that can be borrowed?.

[martenrichter]

Well, first the apple silicon macs are coming to GitHub
github/roadmap#528
however, they do not tell, what the name of the runner is.
The universal binaries are an option, since it seems to use only certain cflags, that should be possible.
However I would prefer, that also the runners are available.
Since I also like to have unit tests on every platform.

[martenrichter]

And yes the node-datachannel also looks promising. I may try to get it running on the weekend.

[martenrichter]

Can you try the current development branch for m1 macs and try npm run build?
It should fail, but I do not get the logs from the GitHub runners (it fails silently).
The remaining problem is, that I also compile a protoc binary for execution on the build host, which can not be arm64 on the GitHub runner, therefore I try to switch to a universal binary.
But CMake fails silently.
The lines, which are related to the problem are:
set_target_properties(protobuf PROPERTIES CMAKE_OSX_ARCHITECTURES x86_64 arm64 CACHE STRING "Build architectures for Mac OS X" FORCE )
if this is fixed and it compiles on your machine, probably the solution will also work for the prebuild.
But I am stuck since I do not have a mac here.

[martenrichter]

#96

[martenrichter]

See also "/Users/runner/work/webtransport/webtransport/build_darwin_x64/CMakeFiles/CMakeOutput.log". See also "/Users/runner/work/webtransport/webtransport/build_darwin_x64/CMakeFiles/CMakeError.log"
is what is interesting.

[achingbrain]

I get:

... more here
-- CMAKE_SYSTEM:             Darwin-21.5.0
-- CMAKE_SYSTEM_NAME:        Darwin
-- CMAKE_SYSTEM_VERSION:     21.5.0
-- CMAKE_SYSTEM_PROCESSOR:   x86_64
-- CMAKE_SKIP_RPATH:         NO
-- CMAKE_VERBOSE_MAKEFILE:   FALSE
-- CMAKE_C_FLAGS:             -Wno-error=deprecated-declarations -Qunused-arguments
-- CMAKE_BUILD_TYPE:         Release
-- CMAKE_C_COMPILER:         /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- CMAKE_AR:                 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar
-- CMAKE_RANLIB:             /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib
-- 
CMake Error at CMakeLists.txt:44 (set_target_properties):
  set_target_properties called with incorrect number of arguments.


-- 
-- 3.21.2.0
CMake Deprecation Warning at third_party/zlib/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Renaming
--     /Users/alex/Documents/Workspaces/achingbrain/webtransport/third_party/zlib/zconf.h
-- to 'zconf.h.included' because this file is included with zlib
-- but CMake generates it automatically in the build directory.
-- Configuring incomplete, errors occurred!
See also "/Users/alex/Documents/Workspaces/achingbrain/webtransport/build_darwin_arm64/CMakeFiles/CMakeOutput.log".
See also "/Users/alex/Documents/Workspaces/achingbrain/webtransport/build_darwin_arm64/CMakeFiles/CMakeError.log".
FAILED: build.ninja 
/usr/local/Cellar/cmake/3.24.2/bin/cmake --regenerate-during-build -S/Users/alex/Documents/Workspaces/achingbrain/webtransport -B/Users/alex/Documents/Workspaces/achingbrain/webtransport/build_darwin_arm64
ninja: error: rebuilding 'build.ninja': subcommand failed
ERR! OMG Process terminated: 1
child process exited with code 1
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "1".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

[achingbrain]

The ERR_UNHANDLED_REJECTION noise is fixed in master by awaiting on the execbuild output

[achingbrain]

Changing:

set_target_properties(protobuf PROPERTIES CMAKE_OSX_ARCHITECTURES x86_64 arm64 CACHE STRING "Build architectures for Mac OS X" FORCE )

to (quotes around "x86_64 arm64")

set_target_properties(protobuf PROPERTIES CMAKE_OSX_ARCHITECTURES "x86_64 arm64" CACHE STRING "Build architectures for Mac OS X" FORCE )

then gives:

CMake Error at CMakeLists.txt:44 (set_target_properties):
  set_target_properties Can not find target to add properties to: protobuf

[martenrichter]

I think I am stuck, problem is still, that protoc is built for arm and should run on intel and I am unable to make protobuf stuff multi-arch.
So I think I will leave it for now, and wait until apple arm GitHub hosted runners are available (I wonder if there is access to the beta program). But of course, if you get it to cross-compile (you can do it the other way round), I will include it, for now, one day is the maximum time to spend for me on this(even though I had to wait most of the time).

[martenrichter]

Some progress, protoc problem is solved. Now, the wrong icu lib is found.

[martenrichter]

Cross-compile is working now. If you have by any chance an intel mac you can try the steps from the action and test the arm binary.
The final test, if every thing works the final test requires a prerelease since the yml file for publishing the package is otherwise not triggered.