Workspaces don't run lifecycle scripts when linking local packages #3911

jquense · 2017-07-11T20:46:35Z

What is the current behavior?

When you do a yarn install in a workspaces enabled project, yarn doesn't run the various lifecycle scripts for workspace packages, e.g. prepare prepublish preinstall etc

If the current behavior is a bug, please provide the steps to reproduce.

Here is a commit adding a failing test case: jquense@61eaf42

What is the expected behavior?

Lifecycle scripts are run for the local packages so they have an opportunity to build steps. This is what lerna does.

Please mention your node.js, yarn and operating system version.

yarn 27.5
node 8
Mac Sierra

cc @bestander

The text was updated successfully, but these errors were encountered:

bestander · 2017-07-11T20:48:46Z

PR is welcome BTW :)

jquense · 2017-07-13T16:23:20Z

I took a stab at it, but could figure out the code base unfortunately. All I can see is that scripts is empty for the linked packages when it gets to the getInstallCommands in the pacakge-install-scripts :/

bestander · 2017-07-13T21:36:28Z

Thanks for giving it a try, @jquense.
The entry point is probably here https://github.com/yarnpkg/yarn/blob/master/src/cli/commands/install.js#L477 - flattenedTopLevelPatterns only contains the top level dependencies of root package.json + a virtual dependency that refers all workspaces.
I think going deeper PackageInstallScripts should handle that https://github.com/yarnpkg/yarn/blob/master/src/package-install-scripts.js#L248.

If you want you can start with submitting a PR with a failing test and we can sort it out in a follow up.

jquense · 2017-07-25T18:38:57Z

The failing test case linked in the issue is the best I can do sorry! Unfortunately I just don't have the time to spend hours learning the yarn codebase to try and fix this.

BurtHarris · 2017-08-01T02:28:28Z

Hi. I'm quite interested in yarn and workspaces in particular. I'll be giving a look at this as a possible first contribution.

bestander · 2017-08-01T05:34:05Z

Awesome! If you have any questions ask away

…

On Mon, Jul 31, 2017 at 7:28 PM Burt Harris ***@***.***> wrote: Hi. I'm quite interested in yarn and workspaces in particular. I'll be giving a look at this as a possible first contribution. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3911 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBdWBmqV2fFlBlOvOlXB77Ml0213Xw6ks5sTo1UgaJpZM4OUyrV> .

BurtHarris · 2017-08-01T07:54:46Z

@bestander I hit a snag right away, Issue #4059. Its almost certainly related to the fact I'm running Windows any hints on how to make that work?

bestander · 2017-08-01T19:53:07Z

As a workaround you should be able to run tests like this node ./node_modules/jest/bin/jest.js

jquense · 2017-08-15T14:25:52Z

Any word from anyone? I might have some time to jump, and I really would love to get this fixed :) I have to think that for most folks the lack of lifecycle scripts running is a dealbreaker for this feature?

bestander · 2017-08-16T00:31:02Z

The core team is focused on fixing the most high priority issues right now, cases where Yarn may crash or install wrong versions of dependencies.

This bug will be resolved eventually but considering that this is a community project the quickest way would be to spend time learning the codebase and sending a fix.

jquense · 2017-08-16T01:18:05Z

Definately :) I'm not trying to be pushy or entitled just wanted to get a sense of where the issue was and if there was any existing work

ryanhiebert · 2017-08-16T16:46:11Z

FWIW, it's not the blocker for my adoption of workspaces. I haven't figured out how to reliable add and remove dependencies (not unlikely to be user error). Handling this was annoying, but I just needed to add calls to the packages' prepare scripts to the parent package.json prepare script, which was a good enough workaround for me.

nkbt · 2017-08-18T00:53:11Z

Some more findings (yarn workspaces + lerna) that might be helpful

With workspaces lerna bootstrap is unnecessary and does nothing, so we just run prepublish after initial install

yarn install --pure-lockfile
yarn lerna run prepublish

Lerna is smart enough to run prepublish in correct order so libs are not getting build before their deps.

The only thing is you should not enforce parallel lerna build like yarn lerna run prepublish -- --parallel. In this case you may get build errors when order matters.

jquense · 2017-09-10T20:15:20Z

I've also found the prepublish lerna approah works well. However the one problem with the lerna run approach is packages that define binaries that other packages use in their prepublish scripts. Lerna runs them in the correct order, but because the the prepublish creates the bin file after the install process is run none of the other local packages have that bin linked in thier .bin folders and so break.

Hypnosphi · 2017-09-10T22:08:17Z

@jquense my workaround was to create additional entries that just require the actual binaries. Those entries are present at the moment of installation, so yarn has something to link.
See storybookjs/storybook@4d10a55#diff-6c1388595b10c4ae5635b68cf08edc98

darkobits · 2017-09-12T17:26:34Z

This behavior is currently making it impossible to migrate from Lerna's bootstrap to Yarn Workspaces.

It is imperative that the prepare/prepublish scripts are run (and in the correct order in each package, based on the dependency graph) to ensure that linked dependencies are built and available to dependents.

bestander · 2017-10-16T00:33:03Z

Sounds reasonable, send a PR

martijnthe · 2017-11-09T19:48:15Z

As a work-around, what seems to work for us is to add the lifecycle scripts to the root package.json and use lerna to "forward" them to the packages in the workspace, so for example:

repo_root/package.json:

"scripts": {
  "prepare": "lerna run --stream --sort prepare"
  ... etc ...
}

If yarn install is run in the root, prepare will get run for all the packages in the workspace.

martijnthe · 2017-11-26T20:26:13Z

Incidentally, I'm now running into: #4973
Is that the same issue perhaps?

The work-around I had posted above ^^^^ doesn't work for packages that aren't part of the workspace.

stipsan · 2018-06-20T11:54:53Z

I hope I don't deter anyone else from starting, so unless I update this thread and say I'm working on it please assume that I'm not working on it 😅

…are" Yarn workspaces currently don't run the npm lifecycle scripts of workspace packages (c.f. yarnpkg/yarn#3911). This is a simple way to get things working for now. Test plan: run `yarn` in the workspace root and see that tsc runs in expo.

NathanielHill · 2019-01-31T14:34:14Z

I'm using @ccapndave technique to trigger prepare in a shared react component library that needs transpilation. Would be nice if #6869 could be merged, even if it's only for prepare and prepublish as there are ordering issues with the others.

thejohnfreeman · 2019-02-09T17:27:27Z

I'm using @martijnthe's solution above and when I yarn, the top-level prepare script runs twice. When I yarn prepare, it only runs once, as expected. Does this happen to anyone else?

  "scripts": {
    "prepare": "lerna run prepare"
  }

FezVrasta · 2019-02-10T03:47:56Z

For everybody in need of this feature. Take a look at lerna-alias. It's a good way to avoid to need to build the workspace packages during developement

Workaround for yarnpkg/yarn#3911

Add prepare lifecycle script to root as a workaround for yarnpkg/yarn#3911

Workaround for yarnpkg/yarn#3911

mike-marcacci · 2019-05-09T02:41:19Z

Came here looking for a different problem, but wanted to note that I don't terribly mind the current state of things, as it allows me to control which packages' scripts get run. Imagine a really, really big monorepo: you wouldn't necessarily want to run every prepare script just to work on one!

If you want to simulate the proposed behavior, you can quite easily do so in the corresponding lifecycle script in the top-level package.json:

{
  // ...
  "scripts": {

    // This runs the `prepare` script in each package.
    "prepare": "for P in packages/*/; do echo \"\n\n---------- $P\" && (cd $P && yarn prepare); done"

    // This runs the `test` script in each package, but exits on the first failure
    "test": "for P in packages/*/; do echo \"\n\n---------- $P\" && if ! (cd $P && yarn test); then exit 1; fi; done"
  }
}

I think any solution that changes the current behavior would need a way to opt out.

Workaround for yarnpkg/yarn#3911

pladaria · 2019-05-29T09:55:58Z

@mike-marcacci if you want to avoid building already built packages, then skip build process if already built

yarn should execute all lifecycle scripts because you don't know what dependencies your package has. The whole repo must be built.

mike-marcacci · 2019-05-29T16:35:19Z

Yes, I think that there is probably a better strategy than an all-or-nothing approach to this (regardless of what the default behavior is). Let's say I have a repo with 26 packages – I may want to work in packages/c which depends on packages/b which depends on packages/a. I would ideally be able to run yarn inside packages/c, and yarn would know to:

install all external dependancies
run the prepare script in packages/a
run the prepare script in packages/b
run the prepare script in packages/c

...without building the remaining 23 packages.

packages/a/package.json
packages/b/package.json
packages/c/package.json
...
packages/z/package.json

package.json

Additionally, running yarn from the project root would be able to traverse the dependency tree and build all packages in the correct order. Without this, running all prepare scripts would be useless in many scenarios (since they may be run out-of-order).

AuthX is one project I've been working on where the build order is important. Note how the build script hard-codes the order of some packages, which are dependancies of the later ones.

dosentmatter · 2019-06-06T19:37:08Z

The root prepare script is still called during yarn install, so you can take advantage of that to call prepare in each of your workspaces.

Easiest way - using yarn workspaces with lerna:
This will run prepare in all workspaces that contain a prepare script. Make sure your dependency graph is correct so lerna can run the prepare scripts in the correct order - the default for lerna is topological (least dependent package runs first).

# package.json
  "scripts": {
    "prepare": "lerna run prepare",
    ...
  },

If you are using yarn and yarn workspaces by itself:
This solution can work, but has a negative where workspaces that do not contain a prepare script will still have prepare run and you will see error Command "prepare" not found. Your yarn install or yarn prepare will exit right at that moment. A workaround is to add a prepare script to all workspaces, even if it's just a NOP.

# package.json
  "scripts": {
    "prepare": "yarn workspaces run prepare",
    ...
  },

Or you can specifically select which workspaces will have prepare run:

# package.json
  "scripts": {
    "prepare": "yarn workspace workspace-a run prepare && yarn workspaces workspace-b run prepare",
    ...
  },

Or you can write a shell script that parses the JSON output from yarn workspaces info. Then you can run yarn workspace $workspace_name run prepare for each workspace. This way, you can ignore errors if the prepare script doesn't exist. I don't think yarn has a node library. But you can call it from node using shelljs for easy JSON parsing.

# package.json
  "scripts": {
    "prepare": "./scripts/prepare.sh",
    ...
  },

ryanhiebert · 2019-06-06T21:03:04Z

yarn workspaces run prepare would be a nice solution for me if I could ensure that sub-dependencies are prepared first. For example, I have package a that depends on packages b and c, and c depends on d. It's important, then, that that d run before c, and b and c both run before a. Apart from that, I don't care what order they run in. It would be really, really nice if yarn would do this for me, ideally, or secondarily give me a command that will list all of the workspaces in a dependecy-deterministic order for this purpose. Parsing these inter-dependencies seems well beyond the scope of what I should be doing in order to write a simple prepare script.

dosentmatter · 2019-06-07T18:46:06Z

@ryanhiebert, is there any reason you don't want to use lerna? It handles the dependency order for you and runs them concurrently. It's also not difficult to add to your project and integrate with yarn workspaces.

How does it [yarn] compare to Lerna?

Yarn’s workspaces are the low-level primitives that tools like Lerna can (and do!) use. They will never try to support the high-level feature that Lerna offers, but by implementing the core logic of the resolution and linking steps inside Yarn itself we hope to enable new usages and improve performance.

I don't think it's worth the development time for yarn to support all the things that lerna run can already do.

Now if someone invokes yarn in the monorepo, or installs a package from a subdirectory with git (e.g. w/ gitpkg.now.sh) the build is automatically invoked. Because of yarnpkg/yarn#3911, we still need a top-level prepare.

itsMapleLeaf · 2020-06-17T19:26:24Z

is there any reason you don't want to use lerna? It handles the dependency order for you and runs them concurrently. It's also not difficult to add to your project and integrate with yarn workspaces.

How does it [yarn] compare to Lerna?

Yarn’s workspaces are the low-level primitives that tools like Lerna can (and do!) use. They will never try to support the high-level feature that Lerna offers, but by implementing the core logic of the resolution and linking steps inside Yarn itself we hope to enable new usages and improve performance.

I don't think it's worth the development time for yarn to support all the things that lerna run can already do.

I'd prefer not to have to pick up, learn, and manage an entire second tool that does a lot more than I need just for one feature. But I can't speak for others.

When using yarn workspaces in lerna the npm lifecycle scripts don't work as intended. In this case running `prepare` in all bootstrapped packages is a step during `lerna bootstrap`. Adding the script in the root package.json solves the problem. More can be read [here](yarnpkg/yarn#3911 (comment)).

Now if someone invokes yarn in the monorepo, or installs a package from a subdirectory with git (e.g. w/ gitpkg.now.sh) the build is automatically invoked. Because of yarnpkg/yarn#3911, we still need a top-level prepare.

ecanchev-jaspersoft · 2024-05-23T19:50:49Z

I cannot use preinstall and put a lerna command because the lerna package is not downloaded when executing the preinstall.
What is the alternative here?

Logs:

2024-05-23 11:51:28.868 | . yarn install v1.22.19
[2](pipeline-console/?start-byte=0&selected-node=84#log-2)
2024-05-23 11:51:28.868 | . $ lerna run --stream --parallel preinstall
[3](/pipeline-console/?start-byte=0&selected-node=84#log-3)
2024-05-23 11:51:28.868 | . /bin/sh: lerna: command not found
[4](/pipeline-console/?start-byte=0&selected-node=84#log-4)
2024-05-23 11:51:28.868 | . error Command failed with exit code 127.
[5](/pipeline-console/?start-byte=0&selected-node=84#log-5)
2024-05-23 11:51:28.868 | . info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[6](/pipeline-console/?start-byte=0&selected-node=84#log-6)
script returned exit code 127

the 'prepare' script is not an option for me as I want to execute a script before the installation of the packages.
I'm coming from lerna bootstrap and want to adopt the workspaces provided by yarn.
yarn version: 1.22.x

bestander added feature-workspaces triaged labels Jul 11, 2017

intellix mentioned this issue Aug 17, 2017

Get example to work sparkles-dev/a-glimpse-at-yarn-workspaces#2

Open

Hypnosphi mentioned this issue Sep 6, 2017

Yarn workspaces storybookjs/storybook#1810

Merged

BYK added cat-bug good first issue help wanted high-priority and removed feature-workspaces labels Oct 27, 2017

TxHawks mentioned this issue Nov 1, 2017

WIP: Yarn workspace robinweser/fela#441

Closed

martijnthe mentioned this issue Nov 26, 2017

Post-install scripts don't execute #4973

Open

rubensworks mentioned this issue Aug 1, 2018

No lifecycle scripts are run anymore comunica/comunica#190

Closed

nsaechao mentioned this issue Aug 10, 2018

Pants should support Node.js source-level dependencies pantsbuild/pants#6332

Closed

rasmuswoelk mentioned this issue Nov 21, 2018

accept function for 'focused' prop folio-org/stripes-react-hotkeys#7

Merged

tomasz-sodzawiczny linked a pull request Jan 3, 2019 that will close this issue

Trigger lifecycle scripts for workspaces #6869

Open

axelchalon mentioned this issue Mar 13, 2019

Parity Ethereum bundling openethereum/fether#458

Merged

markspanbroek added a commit to philips-software/cogito that referenced this issue Apr 16, 2019

[cogito-attestations-ios] Run prepare lifecycle scripts

01f8bc5

Workaround for yarnpkg/yarn#3911

markspanbroek added a commit to philips-software/cogito that referenced this issue Apr 18, 2019

[cogito-attestations-ios] Run prepare lifecycle scripts

fec05dc

Workaround for yarnpkg/yarn#3911

AVykhrystyuk added a commit to AVykhrystyuk/magpie that referenced this issue Apr 18, 2019

chore: fix the build delegating prepare script

ffb09b0

Add prepare lifecycle script to root as a workaround for yarnpkg/yarn#3911

JeroenKnoops pushed a commit to philips-software/cogito that referenced this issue Apr 24, 2019

[cogito-attestations-ios] Run prepare lifecycle scripts

0424035

Workaround for yarnpkg/yarn#3911

markspanbroek added a commit to philips-software/cogito that referenced this issue May 13, 2019

[cogito-attestations-ios] Run prepare lifecycle scripts

e59b640

Workaround for yarnpkg/yarn#3911

scinos mentioned this issue Nov 18, 2020

Use babel to transpile TS projects Automattic/wp-calypso#47502

Merged

kachkaev mentioned this issue Feb 1, 2022

Replace postinstall with prepare in mock-block-dock blockprotocol/blockprotocol#173

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Workspaces don't run lifecycle scripts when linking local packages #3911

Workspaces don't run lifecycle scripts when linking local packages #3911

jquense commented Jul 11, 2017

bestander commented Jul 11, 2017

jquense commented Jul 13, 2017

bestander commented Jul 13, 2017

jquense commented Jul 25, 2017

BurtHarris commented Aug 1, 2017

bestander commented Aug 1, 2017 via email

BurtHarris commented Aug 1, 2017

bestander commented Aug 1, 2017

jquense commented Aug 15, 2017

bestander commented Aug 16, 2017 •

edited

Loading

jquense commented Aug 16, 2017

ryanhiebert commented Aug 16, 2017 •

edited

Loading

nkbt commented Aug 18, 2017

jquense commented Sep 10, 2017

Hypnosphi commented Sep 10, 2017 •

edited

Loading

darkobits commented Sep 12, 2017

bestander commented Oct 16, 2017

martijnthe commented Nov 9, 2017

martijnthe commented Nov 26, 2017

stipsan commented Jun 20, 2018

NathanielHill commented Jan 31, 2019

thejohnfreeman commented Feb 9, 2019 •

edited

Loading

FezVrasta commented Feb 10, 2019

mike-marcacci commented May 9, 2019 •

edited

Loading

pladaria commented May 29, 2019

mike-marcacci commented May 29, 2019 •

edited

Loading

dosentmatter commented Jun 6, 2019 •

edited

Loading

ryanhiebert commented Jun 6, 2019 •

edited

Loading

dosentmatter commented Jun 7, 2019 •

edited

Loading

itsMapleLeaf commented Jun 17, 2020

ecanchev-jaspersoft commented May 23, 2024 •

edited

Loading

Workspaces don't run lifecycle scripts when linking local packages #3911

Workspaces don't run lifecycle scripts when linking local packages #3911

Comments

jquense commented Jul 11, 2017

bestander commented Jul 11, 2017

jquense commented Jul 13, 2017

bestander commented Jul 13, 2017

jquense commented Jul 25, 2017

BurtHarris commented Aug 1, 2017

bestander commented Aug 1, 2017 via email

BurtHarris commented Aug 1, 2017

bestander commented Aug 1, 2017

jquense commented Aug 15, 2017

bestander commented Aug 16, 2017 • edited Loading

jquense commented Aug 16, 2017

ryanhiebert commented Aug 16, 2017 • edited Loading

nkbt commented Aug 18, 2017

jquense commented Sep 10, 2017

Hypnosphi commented Sep 10, 2017 • edited Loading

darkobits commented Sep 12, 2017

bestander commented Oct 16, 2017

martijnthe commented Nov 9, 2017

martijnthe commented Nov 26, 2017

stipsan commented Jun 20, 2018

NathanielHill commented Jan 31, 2019

thejohnfreeman commented Feb 9, 2019 • edited Loading

FezVrasta commented Feb 10, 2019

mike-marcacci commented May 9, 2019 • edited Loading

pladaria commented May 29, 2019

mike-marcacci commented May 29, 2019 • edited Loading

dosentmatter commented Jun 6, 2019 • edited Loading

ryanhiebert commented Jun 6, 2019 • edited Loading

dosentmatter commented Jun 7, 2019 • edited Loading

itsMapleLeaf commented Jun 17, 2020

ecanchev-jaspersoft commented May 23, 2024 • edited Loading

bestander commented Aug 16, 2017 •

edited

Loading

ryanhiebert commented Aug 16, 2017 •

edited

Loading

Hypnosphi commented Sep 10, 2017 •

edited

Loading

thejohnfreeman commented Feb 9, 2019 •

edited

Loading

mike-marcacci commented May 9, 2019 •

edited

Loading

mike-marcacci commented May 29, 2019 •

edited

Loading

dosentmatter commented Jun 6, 2019 •

edited

Loading

ryanhiebert commented Jun 6, 2019 •

edited

Loading

dosentmatter commented Jun 7, 2019 •

edited

Loading

ecanchev-jaspersoft commented May 23, 2024 •

edited

Loading