Add Node cross-compilation support #14

dennisameling · 2021-05-01T23:05:45Z

Successor of #7, needed for signalapp/Signal-Desktop#3745, signalapp/Signal-Desktop#4461 and signalapp/Signal-Desktop#1636.

Allow cross-compiling for NodeJS on Windows, macOS and Linux
Bump Rust toolchain to 1.49.0 to have Apple Silicon support
Drop armv7 iOS builds since they are no longer supported in Rust 1.49.0. Apple has been accepting 64-bit apps since 2015 and has officially dropped support for 32-bit apps (iPhone 5s and later)
Update CI to build releases for all supported architectures (x64, arm64, ia32)
Add crossbuild packages for Ubuntu to allow for Linux cross-compilation builds

Supported architectures:

Windows:
- x64
- ia32
- arm64
Linux:
- x64
- ia32
- arm64
MacOS:
- x64
- arm64 (Apple Silicon)

Tests & release:

Successful test run: https://github.com/dennisameling/zkgroup/actions/runs/804515314
Successful release run: https://github.com/dennisameling/zkgroup/actions/runs/804515312
Test release: https://github.com/dennisameling/zkgroup/releases/tag/v0.7.2-cross-test3
signal-zkgroup-node update including generated binaries: dennisameling/signal-zkgroup-node@5f28c4b

jrose-signal

I'm hesitant to do too much here since we'd like to move zkgroup into libsignal-client (not-now-but-hopefully-soon). Like RingRTC I think it would still make sense not to publish artifacts for platforms the official client apps don't support (yet?); making sure the build works is one thing but publishing artifacts gives the impression that they're supported.

jrose-signal · 2021-05-03T18:02:45Z

rust-toolchain

If we're going to update we may as well update to current stable.

jrose-signal · 2021-05-03T18:25:00Z

.github/workflows/artifacts.yml

@@ -127,22 +170,35 @@ jobs:
          node-version: '${{ steps.node_version.outputs.NODE_VERSION }}'

      - name: Build
+        env:
+          # Needed until hosted macos-11.0 runners become available
+          SDKROOT: /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk


This is no good because they might update to 11.2. Is it sufficient to use xcrun -sdk macosx make below? xcrun automatically sets a bunch of environment variables like SDKROOT but I don't know if it'll pick the latest SDK or the host SDK.

jrose-signal · 2021-05-03T18:31:51Z

.github/workflows/artifacts.yml

+      - name: Upload Linux x64
        uses: svenstaro/upload-release-action@v1-release
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
-          file: ffi/node/libzkgroup.so
-          asset_name: libzkgroup.so
+          file: ffi/node/libzkgroup-x64.so
+          asset_name: libzkgroup-x64.so
+          tag: ${{ github.ref }}
+          overwrite: true
+
+      - name: Upload Linux ia32
+        uses: svenstaro/upload-release-action@v1-release
+        with:
+          repo_token: ${{ secrets.GITHUB_TOKEN }}
+          file: ffi/node/libzkgroup-ia32.so
+          asset_name: libzkgroup-ia32.so
+          tag: ${{ github.ref }}
+          overwrite: true
+
+      - name: Upload Linux arm64
+        uses: svenstaro/upload-release-action@v1-release
+        with:
+          repo_token: ${{ secrets.GITHUB_TOKEN }}
+          file: ffi/node/libzkgroup-arm64.so
+          asset_name: libzkgroup-arm64.so
          tag: ${{ github.ref }}
          overwrite: true


Let's use file_glob: true for these and then we can have a single upload action for all the Linux ones.

dennisameling · 2021-05-04T07:38:28Z

@jrose-signal thanks for reviewing this! You mentioned the following:

Like RingRTC I think it would still make sense not to publish artifacts for platforms the official client apps don't support (yet?)

So does this mean that I should take out Windows arm64/ia32, macOS arm64 and Linux arm64/ia32 from the GH Actions workflow for now? So that we'll only have x64 for now, just like before? That also means that we no longer need the macOS 11 SDK nor do we need the file_glob as you mentioned. It would indeed simplify this PR a lot, but just checking if I understand your remarks correctly! 😊

jrose-signal · 2021-05-05T00:56:05Z

Yep, I think that's the way to go for now (though let's conclude our conversation on the RingRTC one too). When the Signal team decides to build for a new arch, we'll be ready (and of course you can continue to build unofficial clients), but we don't need to modify the default artifacts just yet.

jrose-signal · 2021-05-06T21:00:49Z

ffi/node/Makefile

@@ -1,6 +1,13 @@
 ZKGROUP_RUST_DIR=../../rust
 ZKGROUP_TARGET_DIR=../../target

+NODE_ARCH := $(shell node -p "process.arch" || x64)


|| echo x64?

Good catch! Just tested locally and indeed needs the echo 👍🏼

jrose-signal · 2021-05-06T21:02:34Z

ffi/node/Makefile

+ifdef CARGO_TARGET
+CARGO_BUILDFLAGS=--target $(CARGO_TARGET)
+endif


CARGO_BUILD_TARGET is the standard Rust environment variable for this; might as well use it. (I'm not sure we even need the extra flags if we do.)

Thank you! Was looking how to get the default build target but didn't come across this env variable. Will change accordingly :) indeed don't need the extra flags then

jrose-signal

I'm going to hold off on merging this until I have a chance to test it locally, but this looks good. Thank you for pushing this through!

dennisameling · 2021-05-06T22:04:03Z

Just applied your suggestions, indeed simplifies things further. Now one can simply run e.g. make libzkgroup CARGO_BUILD_TARGET=aarch64-pc-windows-msvc NODE_ARCH=arm64 to generate a Windows arm64 dll 👍🏼

sebdanielsson · 2021-05-22T17:39:50Z

Will this me merged soon? Can't wait for arm64 support on macOS🤤

jrose-signal · 2021-05-24T17:17:57Z

Sorry, it's been busier than expected over here, but I haven't forgotten!

jrose-signal · 2021-06-14T23:04:54Z

For the record, this looks good, and we'll be incorporating it in an upcoming release. I'll comment here when the tag is released!

jrose-signal · 2021-06-25T17:48:05Z

Included in v0.7.3 as 2dc537e. Thank you again!

This was referenced May 1, 2021

Add cross-compilation for x86 and arm64 targets (including Apple Silicon) #7

Closed

Support ARM-based macOS signalapp/Signal-Desktop#4461

Closed

[Feature Request] Compile for Windows 10 on ARM signalapp/Signal-Desktop#3745

Closed

dennisameling force-pushed the node-cross-compilation branch 2 times, most recently from f33b81e to 674d090 Compare May 2, 2021 15:10

dennisameling changed the title ~~[WIP] Add Node cross-compilation support~~ Add Node cross-compilation support May 2, 2021

dennisameling marked this pull request as ready for review May 2, 2021 15:10

dennisameling mentioned this pull request May 3, 2021

Add Electron multi-arch/cross-compilation support signalapp/ringrtc#12

Closed

8 tasks

jrose-signal reviewed May 3, 2021

View reviewed changes

dennisameling force-pushed the node-cross-compilation branch 2 times, most recently from 8467a3a to 2fd2316 Compare May 6, 2021 20:50

jrose-signal reviewed May 6, 2021

View reviewed changes

dennisameling force-pushed the node-cross-compilation branch from 2fd2316 to 02c3866 Compare May 6, 2021 21:18

jrose-signal approved these changes May 6, 2021

View reviewed changes

Add Node cross-compilation support

8f72e97

dennisameling force-pushed the node-cross-compilation branch from 02c3866 to 8f72e97 Compare May 6, 2021 22:02

jrose-signal closed this Jun 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Node cross-compilation support #14

Add Node cross-compilation support #14

dennisameling commented May 1, 2021 •

edited

Loading

jrose-signal left a comment

jrose-signal May 3, 2021

jrose-signal May 3, 2021

jrose-signal May 3, 2021

dennisameling commented May 4, 2021

jrose-signal commented May 5, 2021

jrose-signal May 6, 2021

dennisameling May 6, 2021

jrose-signal May 6, 2021

dennisameling May 6, 2021

jrose-signal left a comment

dennisameling commented May 6, 2021

sebdanielsson commented May 22, 2021

jrose-signal commented May 24, 2021

jrose-signal commented Jun 14, 2021

jrose-signal commented Jun 25, 2021

Add Node cross-compilation support #14

Add Node cross-compilation support #14

Conversation

dennisameling commented May 1, 2021 • edited Loading

Supported architectures:

Tests & release:

jrose-signal left a comment

Choose a reason for hiding this comment

jrose-signal May 3, 2021

Choose a reason for hiding this comment

jrose-signal May 3, 2021

Choose a reason for hiding this comment

jrose-signal May 3, 2021

Choose a reason for hiding this comment

dennisameling commented May 4, 2021

jrose-signal commented May 5, 2021

jrose-signal May 6, 2021

Choose a reason for hiding this comment

dennisameling May 6, 2021

Choose a reason for hiding this comment

jrose-signal May 6, 2021

Choose a reason for hiding this comment

dennisameling May 6, 2021

Choose a reason for hiding this comment

jrose-signal left a comment

Choose a reason for hiding this comment

dennisameling commented May 6, 2021

sebdanielsson commented May 22, 2021

jrose-signal commented May 24, 2021

jrose-signal commented Jun 14, 2021

jrose-signal commented Jun 25, 2021

dennisameling commented May 1, 2021 •

edited

Loading