Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug?]: Fail to install turbo when using nodeLinker: pnpm with v4 #5165

Closed
1 task
nutstick opened this issue Dec 31, 2022 · 14 comments · Fixed by #6066
Closed
1 task

[Bug?]: Fail to install turbo when using nodeLinker: pnpm with v4 #5165

nutstick opened this issue Dec 31, 2022 · 14 comments · Fixed by #6066
Labels
bug Something isn't working reproducible This issue can be successfully reproduced

Comments

@nutstick
Copy link

Self-service

  • I'd be willing to implement a fix

Describe the bug

After upgrading to yarn v4 (canary), when run yarn install will lead to the following error on turbo.

# This file contains the result of Yarn building a package (turbo@npm:1.5.5)
# Script name: postinstall

node:internal/errors:863
  const err = new Error(message);
              ^

Error: Command failed: node /node_modules/.store/turbo-npm-1.5.5-30714db15d/package/bin/turbo --version
/node_modules/.store/turbo-npm-1.5.5-30714db15d/package/bin/turbo:13
    throw e;
    ^

<ref *1> Error: spawnSync /node_modules/.store/turbo-darwin-arm64-npm-1.5.5-80deeb5941/package/bin/turbo EACCES
    at Object.spawnSync (node:internal/child_process:1110:20)
    at spawnSync (node:child_process:857:24)
    at Object.execFileSync (node:child_process:900:15)
    at Object.<anonymous> (/node_modules/.store/turbo-npm-1.5.5-30714db15d/package/bin/turbo:6:28)
    at Module._compile (node:internal/modules/cjs/loader:1159:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
    at Module.load (node:internal/modules/cjs/loader:1037:32)
    at Module._load (node:internal/modules/cjs/loader:878:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:23:47 {
  errno: -13,
  code: 'EACCES',
  syscall: 'spawnSync /node_modules/.store/turbo-darwin-arm64-npm-1.5.5-80deeb5941/package/bin/turbo',
  path: '/node_modules/.store/turbo-darwin-arm64-npm-1.5.5-80deeb5941/package/bin/turbo',
  spawnargs: [ '--version' ],
  error: [Circular *1],
  status: null,
  signal: null,
  output: null,
  pid: 26749,
  stdout: null,
  stderr: null
}

When using yarn v3, can be install successfully.

To reproduce

const {promises: {writeFile}} = require(`fs`);

// Enable `pnpm` nodeLinker
await writeFile('.yarnrc.yml', 'nodeLinker: pnpm');

await packageJson({
  devDependencies: {
    [`turbo`]: `^1.5.5`,
  }
});

await expect(yarn(`install`)).rejects.not.toThrow(`Command failed`);;

Environment

System:
    OS: macOS 12.3.1
    CPU: (10) arm64 Apple M1 Max
  Binaries:
    Node: 18.12.1 - /private/var/folders/bl/b4w8tgms0qdf3g_dp8_8nspr0000gn/T/xfs-cb43a611/node
    Yarn: 4.0.0-rc.34 - /private/var/folders/bl/b4w8tgms0qdf3g_dp8_8nspr0000gn/T/xfs-cb43a611/yarn
    npm: 8.19.2 - ~/.nvm/versions/node/v18.12.1/bin/npm

Additional context

No response

@nutstick nutstick added the bug Something isn't working label Dec 31, 2022
@yarnbot yarnbot added the reproducible This issue can be successfully reproduced label Dec 31, 2022
@yarnbot
Copy link
Collaborator

yarnbot commented Dec 31, 2022

This issue reproduces on master:

Error: expect(received).rejects.not.toThrow(expected)

Expected substring: not "Command failed"
Received message:       "Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js install
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: EXDEV: cross-device link not permitted, link '/github/home/.yarn/berry/index/d7/d79c96e67f5cae5a89b54b4c048871ee6e5238e7.dat' -> '/tmp/tmp-18P9n75cM8XZ4E/node_modules/.store/turbo-darwin-64-npm-1.6.3-ced0c215dc/package/package.json'] {
  errno: -18,
  code: 'EXDEV',
  syscall: 'link',
  path: '/github/home/.yarn/berry/index/d7/d79c96e67f5cae5a89b54b4c048871ee6e5238e7.dat',
  dest: '/tmp/tmp-18P9n75cM8XZ4E/node_modules/.store/turbo-darwin-64-npm-1.6.3-ced0c215dc/package/package.json'
}

➤ YN0000: ┌ Resolution step
::group::Resolution step
::endgroup::
➤ YN0000: └ Completed in 0s 835ms
➤ YN0000: ┌ Fetch step
::group::Fetch step
➤ YN0013: │ 5 packages were already cached, 2 had to be fetched
::endgroup::
➤ YN0000: └ Completed in 2s 696ms
➤ YN0000: ┌ Link step
::group::Link step
➤ YN0001: │ Error: EXDEV: cross-device link not permitted, link '/github/home/.yarn/berry/index/d7/d79c96e67f5cae5a89b54b4c048871ee6e5238e7.dat' -> '/tmp/tmp-18P9n75cM8XZ4E/node_modules/.store/turbo-darwin-64-npm-1.6.3-ced0c215dc/package/package.json'
::endgroup::
➤ YN0000: └ Completed
➤ YN0000: Failed with errors in 3s 718ms
"

      at ChildProcess.exithandler (node:child_process:400:12)
      at ChildProcess.emit (node:events:513:28)
      at maybeClose (node:internal/child_process:1093:16)
      at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)
    at Object.toThrow (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-44ff9ab1e7.zip/node_modules/expect/build/index.js:242:20)
    at module.exports (evalmachine.<anonymous>:13:43)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-286d94b96d.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:57:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-286d94b96d.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:18:16)
    at async executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-286d94b96d.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:25:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-286d94b96d.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:26:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-4061026d74.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-4061026d74.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-4061026d74.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

@georeith
Copy link

+1 same issue with esbuild

@7rulnik
Copy link
Contributor

7rulnik commented Mar 15, 2023

I bisected commits and discover that it was introduced in #4542
But I can't say what is wrong with that. @larixer maybe you have any ideas?

@7rulnik
Copy link
Contributor

7rulnik commented Mar 15, 2023

Also @nutstick your reproduction is incorrect. I guess it's because of docker volumes.
Please try to add globalFolder to config

UPD: looks like codesandbox changed how they work with volumes so no need for globalFolder, but this is a bit more accurate I think:

const {promises: {writeFile}} = require(`fs`);

// Enable `pnpm` nodeLinker
await writeFile('.yarnrc.yml', 'nodeLinker: pnpm');

await packageJson({
  devDependencies: {
    [`esbuild`]: `0.17.11`,
  }
});

try {
  await yarn('install')
} catch (err) {
  console.error(err)
}

await expect(yarn('esbuild')).rejects.not.toThrow('EACCES')

@7rulnik
Copy link
Contributor

7rulnik commented Mar 15, 2023

And kinda obvious thing, but still want to mention that we are facing it because of different file permissions:

Works:

ls -l node_modules/.store/@esbuild-darwin-arm64-npm-0.16.17-96c7a38090/node_modules/@esbuild/darwin-arm64/bin
total 17304
-rwxr-xr-x  1 v7rulnik  staff  8858802 23 июн  1984 esbuild*

Doesn't work:

ls -l node_modules/.store/@esbuild-darwin-arm64-npm-0.16.17-96c7a38090/package/bin/
total 17304
-rw-r--r--  2 v7rulnik  staff  8858802 23 июн  1984 esbuild

@7rulnik
Copy link
Contributor

7rulnik commented Mar 15, 2023

I made a mistake here #5165 (comment). For some reason, the error doesn't happen every time so it's better to try to execute the binary directly. So I checked once again and the real problem is content addressing which was introduced in #4586.

So the main problem is that we don't reach chmod setting https://github.com/yarnpkg/berry/blob/master/packages/yarnpkg-fslib/sources/algorithms/copyPromise.ts#L109-L112 when HardlinkFromIndex is used.

@arcanis looks like we should do something different there

@7rulnik
Copy link
Contributor

7rulnik commented Mar 15, 2023

Also, if I change globalFolder index remains empty.
UPD: looks like it because file with same inode already existed in previous globalFolder

@7rulnik
Copy link
Contributor

7rulnik commented Mar 16, 2023

@nutstick @georeith if you want you can install version without content addressing

yarn set version from sources --repository [email protected]:7rulnik/berry.git --branch fix-pnpm

If you are worried about security you can check content via master...7rulnik:berry:fix-pnpm

@shinebayar-g
Copy link

Running into the same error with esbuild installation when using yarn-4.0.0-rc.49 with nodeLinker: pnpm.
pnp & node-modules are working fine.
nodeLinker: pnpm is working fine on yarn 3.6.2 for esbuild installation.

@keepitsimple
Copy link

Bumped into the same issue with vite (it uses esbuild under the hood by default)
esbuild doesn't have the execution bit, in the result, vite doesn't work at all

Could you please fix this issue in pnpn mode in Yarn 4?

@arcanis
Copy link
Member

arcanis commented Nov 1, 2023

I'll look at that later this week.

black7375 added a commit to mincho-js/mincho that referenced this issue Nov 7, 2023
Commands:
```
yarn init -2 -w
yarn set version 3.6.4
yarn config set nodeLinker pnpm
yarn plugin import interactive-tools
yarn plugin import https://mskelton.dev/yarn-outdated/v3
```

Currently, Vite don't works with yarn 4
- yarnpkg/berry#5165
@ptessier
Copy link

👀

@kherock
Copy link
Contributor

kherock commented Jan 6, 2024

I've just run into this with esbuild after trying to upgrade to Yarn 4 - version 3.7.0 still functions as expected. @arcanis, did you find anything earlier?

@fancyfractal
Copy link
Contributor

fancyfractal commented Jan 8, 2024

Note: it's necessary to rm -rf node_modules .yarn/install-state.tgz for the fix to take effect

arcanis added a commit that referenced this issue Jan 23, 2024
…#6066)

**What's the problem this PR addresses?**

This is the fix for missing executable perms when installing
dependencies like turbo and esbuild with pnpm linker
(#5165,
#5991)

Resolves #5165

<!-- Describe the rationale of your PR. -->
<!-- Link all issues that it closes. (Closes/Resolves #xxxx.) -->


**How did you fix it?**
<!-- A detailed description of your implementation. -->
Looks like the code was missing for setting permissions for the file
copied by pnpm linker into content-addressable store

I added `destinationFs.chmodPromise` to `copyFileViaIndex` in `fslib` if
the file is not in store (`if (!indexStat)` branch)

**Checklist**
<!--- Don't worry if you miss something, chores are automatically
tested. -->
<!--- This checklist exists to help you remember doing the chores when
you submit a PR. -->
<!--- Put an `x` in all the boxes that apply. -->
- [x] I have read the [Contributing
Guide](https://yarnpkg.com/advanced/contributing).

<!-- See
https://yarnpkg.com/advanced/contributing#preparing-your-pr-to-be-released
for more details. -->
<!-- Check with `yarn version check` and fix with `yarn version check
-i` -->
- [x] I have set the packages that need to be released for my changes to
be effective.

<!-- The "Testing chores" workflow validates that your PR follows our
guidelines. -->
<!-- If it doesn't pass, click on it to see details as to what your PR
might be missing. -->
- [x] I will check that all automated PR checks pass before the PR gets
reviewed.

---------

Co-authored-by: Maël Nison <[email protected]>
turadg added a commit to Agoric/agoric-3-proposals that referenced this issue Jan 25, 2024
to work around yarnpkg/berry#5165 until 4.0.3 is released
turadg added a commit to Agoric/agoric-3-proposals that referenced this issue Jan 25, 2024
to work around yarnpkg/berry#5165 until 4.0.3 is released
turadg added a commit to Agoric/agoric-3-proposals that referenced this issue Jan 25, 2024
to work around yarnpkg/berry#5165 until 4.0.3 is released
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working reproducible This issue can be successfully reproduced
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants