Flakiness caused by symlink_node_modules

[khogeland]

This may be the same issue as #2014, I'm not sure. On our Windows CI machines, in a build with multiple yarn_install repos, there appears to be some sort of race condition, as ~half our builds fail with errors like:

[12:02:46]W:	 [Step 10/12] ERROR: D:/_bzl/3pjnhl6q/external/loom-ui_yarn/webpack-cli/bin/BUILD.bazel:9:14: Creating runfiles tree bazel-out/host/bin/external/loom-ui_yarn/webpack-cli/bin/webpack-cli.bat.runfiles [for host] failed: missing input file 'external/loom-ui_yarn/node_modules/webpack/lib/dependencies/ContextDependencyTemplateAsId.js', owner: '@loom-ui_yarn//:node_modules/webpack/lib/dependencies/ContextDependencyTemplateAsId.js'
[12:02:46]W:	 [Step 10/12] ERROR: D:/_bzl/3pjnhl6q/external/loom-ui_yarn/webpack-cli/bin/BUILD.bazel:9:14: Creating runfiles tree bazel-out/host/bin/external/loom-ui_yarn/webpack-cli/bin/webpack-cli.bat.runfiles [for host] failed: missing input file 'external/loom-ui_yarn/node_modules/webpack/lib/dependencies/ContextDependencyTemplateAsRequireCall.js', owner: '@loom-ui_yarn//:node_modules/webpack/lib/dependencies/ContextDependencyTemplateAsRequireCall.js'

Where some several thousand input files are missing from node_modules. I haven't seen this happen on our Mac machines.

Setting symlink_node_modules = False on all our yarn_installs seems to solve this issue. Perhaps there is some interplay between multiple yarn repos, sandboxing, and/or and bazelbuild/bazel#8487. Nothing on our end would be deleting/modifying the contents of anything in node_modules. I noticed a similar issue alluded to in one of your example packages, where you've used the same workaround:

https://github.com/bazelbuild/rules_nodejs/blob/90dc2c2f6f7adf46e051e4c8e3c050279b36f45d/examples/angular_bazel_architect/WORKSPACE#L28-L32

As it would be difficult to share a repro, I'm just opening this issue as suggestion for an expert to proofread the implementation of symlink_node_modules for sanity. Especially under various setups: mac/windows, sandboxing on/off, multiple yarn_installs in the same build, etc.

[chris-bannister-privitar]

I am seeing this on Linux machines too

[pcj]

Also flaky on mac, observing same behavior

[alexeagle]

My guess is that we should deprecate and abandon this feature. I recently added a warning on it about this known incorrectness, https://github.com/bazelbuild/rules_nodejs/pull/2946/files
but it's probably really hard to fix, no one has had time, and if we get a better way for bazel to install individual node modules and then link them pnpm-style it won't have an all-of-node_modules tree to share with the users workspace directory anyway.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 6 months. It will be closed if no further activity occurs in 30 days. Collaborators can add a "cleanup" or "need: discussion" label to keep it open indefinitely. Thanks for your contributions to rules_nodejs!

[github-actions]

This issue was automatically closed because it went 30 days without any activity since it was labeled "Can Close?"