Fetching nodes fails on ARM Linux

[marvi]

When using this plugin on ARM Linux (and when node is released for Apple Silicon it will fail there too). It tries to download
-aarch64.tar.gz but the file is named -arm64.tar.gz.

Caused by: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find org.nodejs:node:12.18.1.
    Searched in the following locations:
      - https://nodejs.org/dist/v12.18.1/node-v12.18.1-linux-aarch64.tar.gz

[deepy]

Does this behave the same on 2.x and 3.0.0-rc5? I'm wondering if the conditional might've broken during the switch to kotlin.

We do have this section PlatformHelper.kt#L21-L24 that from memory should be solving this.

But we also only have a single test for this PlatformHelperTest.groovy#L38-L54

[marvi]

I noticed it with 2.2.4 when I had to use ARM Linux with Docker on my Apple Silicon Air. But I also checked out master on a ARM CentOS machine and get the same issue there when running gradlew test. The output above is from the tests when running on CentOS 8/aarch64. I don't know Kotlin so it was a bit hard for me to debug. uname -m returns arm64 but System.getProperty("os.arch") returns aarch64. Could that be a factor? I see that uname is called in PlatformHelper.kt.

I didn't have a local nodejs on the CentOS machine so there where a lot of tests failing. Installing it and running the tests again to try to pinpoint.

[marvi]

com.github.gradle.node.yarn.task.YarnInstall_integTest > install packages with yarn on npm >= 7[0] FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure: Unexpected build execution failure in /tmp/junit14860472788962526274 with arguments [yarn]

    Output:
    > Task :nodeSetup FAILED

    FAILURE: Build failed with an exception.

    * What went wrong:
    Execution failed for task ':nodeSetup'.
    > Could not resolve all files for configuration ':detachedConfiguration1'.
       > Could not find org.nodejs:node:15.2.1.
         Searched in the following locations:
           - https://nodejs.org/dist/v15.2.1/node-v15.2.1-linux-aarch64.tar.gz
         Required by:
             project :

[deepy]

I'll reinstall my raspberry pi at some point during this week which should make it easy to troubleshoot.

Might even be able to add it as a runner to GitHub actions to keep it working in the future

[tompson]

Having the same issue when running on M1 Apple Silicon:

* What went wrong:
Execution failed for task ':nodeSetup'.
> Could not resolve all files for configuration ':detachedConfiguration1'.
   > Could not find node-12.14.1-darwin-x86_64.tar.gz (org.nodejs:node:12.14.1).
     Searched in the following locations:
         https://nodejs.org/dist/v12.14.1/node-v12.14.1-darwin-x86_64.tar.gz

[marvi]

Node uses x64 as architecture identifier instead of x86_64: node-v12.14.1-darwin-x64.tar.gz

[tompson]

yeah, I now, but is there a way to configure the gradle plugin so that it downloads the correct version - it seems to add the x86_64 part on its own

[deepy]

We're using the identifier from java, I spent about an hour yesterday trying to get it to build on my RPi but I don't think I've got enough RAM on that machine for anything meaningful.

@tompson do you know what java says os.arch is on M1?
java -XshowSettings:properties -version and out of curiosity, what's os.name? It's being detected correctly.
And also, will the x64 macOS node work on the M1 macs?

Having added the following to the test:

osProp      | archProp  | osName   | osArch   | isWindows
'Linux'     | 'aarch64' | 'linux'  | 'arm64'  | false

and forcing the uname -m condition to return arm64 (due to temporarily being on windows)
arch == "arm" || arch.startsWith("aarch") -> "arm64".mapIf({ it == "armv8l" }) { "arm64" }

I get the expecfted result: arm64 as osArch and since the code that's deciding which artifact to ask uses this:

fun computeArchiveDependency(nodeExtension: NodeExtension): Provider<String> {
    val osName = platformHelper.osName
    val osArch = platformHelper.osArch
    val type = if (platformHelper.isWindows) "zip" else "tar.gz"
    return nodeExtension.version.map { version -> "org.nodejs:node:$version:$osName-$osArch@$type" }
}

We should be golden, even though we aren't. I'll make a second try on getting the RPi working after lunch.

[deepy]

Something that would've been nice to have is the ability to pass a closure/action of our own that does the platform checking instead of having it hard-coded in the plugin, especially if you're on something like Alpine Linux which uses musl or if you've got your own internal site with nodejs binaries.

[tompson]

unfortunately I do not have access to the M1 MacBook currently but will test it a soon as I have it back

[deepy]

My raspberry pi says os.arch is arm and uname -m says armv7l so the build works fine there, so at least it works in one place

$ ./examples/simple-node/.gradle/nodejs/node-v12.18.1-linux-armv7l/bin/node --version
v12.18.1

[deepy]

This is driving me insane, at some point during the week I'm going to launch an ARM VM and try it there, I've increased the tests in #132 and that returns as expected when I run on my machine :-)

[deepy]

Alright, being back into normal routine and with access to a test machine, it can only be because of uname -m returning aarch64 so we can probably solve this by adding that to the mapping.

@marvi the easiest way to test this is by checking out the arm-tests branch, changing to download = true in examples/simple-node and running ./gradlew -p examples/simple-node helloWorld from the root.

[marvi]

On my Apple Silicon Mac I think it would work, if there was a node build available:

Could not find org.nodejs:node:12.18.1.
     Searched in the following locations:
       - https://nodejs.org/dist/v12.18.1/node-v12.18.1-darwin-arm64.tar.gz

On ARM Linux (Centos 8) running in Docker on Apple Silicon it fetches the correct binary:

# /root/gradle-node-plugin/examples/simple-node/.gradle/nodejs/node-v12.18.1-linux-arm64/bin/node
Welcome to Node.js v12.18.1.

simple-node fails because it's looking for npm in the wrong place:

[org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter] Executing actions for task ':npm:npmInstall'.
[org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Execute exec for :npm:npmInstall' started
[org.gradle.process.internal.DefaultExecHandle] Starting process 'command 'npm''. Working directory: /root/gradle-node-plugin/examples/simple-node/npm Command: npm install 
[org.gradle.process.internal.DefaultExecHandle] Process 'command 'npm'' finished with exit value -1 (state: FAILED)

But maybe this is expected for now? The issue with finding the right binary package seems fixed.

[deepy]

That's probably a different problem yeah, the important part here is that it's fetching the right binary :-)

I've merged this to master and there'll either by a 3.0 released next or a new rc

[tompson]

So, back on my M1 MacBook

Using Azul Zulu java jdk8.0.275-macos_aarch64 outputs the following:

java -XshowSettings:properties -version

os.arch = aarch64
os.name = Mac OS X
os.version = 11.1

[deepy]

That's good, then it'll use the result from uname -m and if that's something weird we can fix that when there's arm builds for macOS available

[aecio]

I'm also seeing this problem when building a docker image with multi-arch support (linux/arm64) using Docker's buildx and QEMU on GitHub Actions.

If not too much work, could you please publish a new release candidate so that we can test the fix? Thanks!

[deepy]

The plan was to release 3.0.0 itself, but there's a bunch of documentation that needs to be written before that, so there's a RC6 that's building released now

[aecio]

I re-run the docker build (on QEMU) using version 3.0.0-rc6 and it worked fine.
Thanks very much, @deepy!

[martin-g]

I've faced this issue on Ubuntu ARM64.
I've updated the plugin version to 3.0.0-rc6 but now it complains with this:

./gradlew nodeSetup

FAILURE: Build failed with an exception.

* Where:
Build file '/home/ubuntu/git/spring/sagan/sagan-client/build.gradle' line: 23

* What went wrong:
A problem occurred evaluating project ':sagan-client'.
> Could not get unknown property 'npm_run_build' for project ':sagan-client' of type org.gradle.api.Project.

build.gradle could be found at: https://github.com/spring-io/sagan/blob/2861286e0fa78a76d5dd543a3459f7e42198b191/sagan-client/build.gradle

Do I need to migrate this build file somehow ?

[deepy]

@martin-g there were some things in the build file that could be improved, I signed the CLA and sent a small PR.

Though using the gradle rule to automatically create npm_foo tasks should still work, I'm pretty sure we even have tests for that specific functionality

[bartoliniii]

I have the same issue now, is there any workaround?

Could not determine the dependencies of task ':nodeSetup'.
> Failed to query the value of task ':nodeSetup' property 'nodeArchiveFile'.
   > Could not resolve all files for configuration ':detachedConfiguration1'.
      > Could not find org.nodejs:node:12.13.0.
        Searched in the following locations:
          - file:/Users/bartek/.m2/repository/org/nodejs/node/12.13.0/node-12.13.0.pom
          - https://repo.maven.apache.org/maven2/org/nodejs/node/12.13.0/node-12.13.0.pom
          - https://nodejs.org/dist/v12.13.0/node-v12.13.0-darwin-arm64.tar.gz
        Required by:
            project :

[deepy]

This is about the Linux support and was fixed a long time ago