Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow symlinks in MountFS #5474

Closed
wants to merge 2 commits into from
Closed

Conversation

thatsmydoing
Copy link
Contributor

@thatsmydoing thatsmydoing commented May 31, 2023

What's the problem this PR addresses?

Yarn shouldn't care if zips in the cache are symlinks or not. The only reason it doesn't work is that finding the mount point skips zips if they're not regular files. This used to be supported until it was removed for "performance" reasons in #1474. The bulk of the logic in the linked PR involves resolving the real path of the symlink but I don't think that's needed for this?

Resolves #3514

How did you fix it?
I switched the check to use stat from lstat.

Checklist

  • I have set the packages that need to be released for my changes to be effective.
  • I will check that all automated PR checks pass before the PR gets reviewed.

@merceyz merceyz self-requested a review May 31, 2023 22:38
@arcanis
Copy link
Member

arcanis commented Jun 1, 2023

The tests seem to accept that, so I'd tend to be fine with merging this since it shouldn't impact existing codebases 🤔

Copy link
Member

@merceyz merceyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This used to be supported until it was removed for "performance" reasons in #1474

Removing support for symlinks wasn't the goal of that PR and since no tests failed the regression wasn't spotted.

Please add a test so we can avoid future regressions, I see the linked issue contains one which can be added to https://github.com/yarnpkg/berry/blob/e20f8369510e2c1c9e82cae2bef7d96e05dbffed/packages/acceptance-tests/pkg-tests-specs/sources/pnp.test.js

@thatsmydoing thatsmydoing force-pushed the allow-symlink-zip branch 4 times, most recently from 19a5d76 to 79c9f98 Compare June 2, 2023 03:23
@thatsmydoing thatsmydoing requested a review from merceyz June 2, 2023 04:03
@thatsmydoing
Copy link
Contributor Author

I've added the tests. The acceptance tests were timing out on windows so I just excluded them. For some reason, the unit test works though so I'm just letting it run unconditionally.

@hansottowirtz
Copy link

hansottowirtz commented Jun 9, 2023

Side note: I'm getting this error on Vercel when setting YARN_CACHE_FOLDER=node_modules/.yarn-cache:

YN0001: │ Error: While persisting /vercel/path0/node_modules/.yarn-cache/node-gyp-npm-9.3.1-43540bab9c-b860e9976f.zip/node_modules/node-gyp/ -> /vercel/path0/node_modules/node-gyp ENOENT: no such file or directory, scandir '/vercel/path0/node_modules/.yarn-cache/node-gyp-npm-9.3.1-43540bab9c-b860e9976f.zip/node_modules/node-gyp'

I hope this PR will also fix that, if not I'll make a new issue.
I have tested this PR's artifacts and it seems to resolve this as well.

@thatsmydoing
Copy link
Contributor Author

@merceyz are there any other changes you want me to make?

@hansottowirtz
Copy link

@merceyz Sorry for pinging, but could this be merged or are there any blockers?

@hansottowirtz
Copy link

@thatsmydoing would you be able to rebase on master?

@hansottowirtz
Copy link

@arcanis Any way this could be merged?

@gabrielrussoc
Copy link

Hey @thatsmydoing @merceyz -- is this safe to be merged?

@hansottowirtz
Copy link

@thatsmydoing I have asked on the Discord, and the Yarn team would need permissions to update your fork: https://discord.com/channels/226791405589233664/1297852972713377863

@thatsmydoing
Copy link
Contributor Author

It seems I can't grant permissions since I've forked it in an org and not my personal account. I can go and rebase this on master though.

github-merge-queue bot pushed a commit that referenced this pull request Nov 18, 2024
This is the same as #5474 but
allows edits by maintainer

**What's the problem this PR addresses?**
<!-- Describe the rationale of your PR. -->
Yarn shouldn't care if zips in the cache are symlinks or not. The only
reason it doesn't work is that finding the mount point skips zips if
they're not regular files. This used to be supported until it was
removed for "performance" reasons in
#1474. The bulk of the logic in the
linked PR involves resolving the real path of the symlink but I don't
think that's needed for this?

<!-- Link all issues that it closes. (Closes/Resolves #xxxx.) -->
Resolves #3514
Closes #5474 (supersedes it)

**How did you fix it?**
I switched the check to use `stat` from `lstat`.

**Checklist**
<!--- Don't worry if you miss something, chores are automatically
tested. -->
<!--- This checklist exists to help you remember doing the chores when
you submit a PR. -->
<!--- Put an `x` in all the boxes that apply. -->
- [X] I have read the [Contributing
Guide](https://yarnpkg.com/advanced/contributing).

<!-- See
https://yarnpkg.com/advanced/contributing#preparing-your-pr-to-be-released
for more details. -->
<!-- Check with `yarn version check` and fix with `yarn version check
-i` -->
- [X] I have set the packages that need to be released for my changes to
be effective.

<!-- The "Testing chores" workflow validates that your PR follows our
guidelines. -->
<!-- If it doesn't pass, click on it to see details as to what your PR
might be missing. -->
- [x] I will check that all automated PR checks pass before the PR gets
reviewed.

---------

Co-authored-by: merceyz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug?]: Yarn fails to handle when cacheFolder is a symlink farm
5 participants