Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packages: Automate npm publishing as part of Gutenberg release workflow #39259

Merged
merged 4 commits into from
Mar 9, 2022
Merged

Packages: Automate npm publishing as part of Gutenberg release workflow #39259

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f0fea36
Packages: Apply changes to npm publishing workflow
gziolo Mar 7, 2022
f74fdf3
Extend plugin release workflow with npm publishing
gziolo Mar 8, 2022
d7c77dc
Set test npm publishing workflow to trigger when Gutenberg RC1 is out
gziolo Mar 9, 2022
c08c754
Update documentation with the reasoning behind Gutenberg plugin sync
gziolo Mar 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/upload-release-to-plugin-repo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,45 @@ jobs:
name: changelog ${{ matrix.label }}
path: ./changelog.txt

npm-publish:
name: Publish WordPress packages to npm
runs-on: ubuntu-latest
needs: update-changelog
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's put npm access token into its own enviroment:

 environment: npm organization

if: ${{ github.event.release.prerelease && endsWith( github.event.release.tag_name, '-rc.1' ) && github.event.release.assets[0] }}
steps:
- name: Checkout (for CLI)
uses: actions/checkout@5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f # v2.3.4
with:
path: main
ref: trunk

- name: Checkout (for publishing)
uses: actions/checkout@5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f # v2.3.4
with:
path: publish
ref: trunk
token: ${{ secrets.GUTENBERG_TOKEN }}

- name: Configure git user name and email (for publishing)
run: |
cd publish
git config user.name "Gutenberg Repository Automation"
git config user.email [email protected]

- name: Setup Node
uses: actions/setup-node@38d90ce44d5275ad62cc48384b3d8a58c500bb5f # v2.2.2
with:
node-version: 14
registry-url: 'https://registry.npmjs.org'

- name: Publish packages to npm ("next" dist-tag)
run: |
cd main
npm ci
./bin/plugin/cli.js npm-next --semver minor --ci --repository-path ../publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

upload:
name: Upload Gutenberg Plugin
runs-on: ubuntu-latest
Expand Down
7 changes: 7 additions & 0 deletions bin/plugin/cli.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,17 @@ const { runPerformanceTests } = require( './commands/performance' );

const semverOption = [ '--semver <semver>', 'Semantic Versioning', 'patch' ];
const ciOption = [ '-c, --ci', 'Run in CI (non interactive)' ];
const repositoryPathOption = [
'--repository-path <repository-path>',
'Relative path to the git repository.',
];

program
.command( 'publish-npm-packages-latest' )
.alias( 'npm-latest' )
.option( ...semverOption )
.option( ...ciOption )
.option( ...repositoryPathOption )
.description(
'Publishes packages to npm (latest dist-tag, production version)'
)
Expand All @@ -45,6 +50,7 @@ program
.alias( 'npm-bugfix' )
.option( ...semverOption )
.option( ...ciOption )
.option( ...repositoryPathOption )
.description(
'Publishes bugfixes for packages to npm (latest dist-tag, production version)'
)
Expand All @@ -55,6 +61,7 @@ program
.alias( 'npm-next' )
.option( ...semverOption )
.option( ...ciOption )
.option( ...repositoryPathOption )
.description(
'Publishes packages to npm (next dist-tag, prerelease version)'
)
Expand Down
55 changes: 37 additions & 18 deletions bin/plugin/commands/packages.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ const {
runCleanLocalFoldersStep,
} = require( './common' );
const git = require( '../lib/git' );
const { join } = require( 'path' );

/**
* Release type names.
Expand All @@ -36,8 +37,9 @@ const git = require( '../lib/git' );
/**
* @typedef WPPackagesCommandOptions
*
* @property {SemVer} [semver] The selected semantic versioning. Defaults to `patch`.
* @property {boolean} [ci] Disables interactive mode when executed in CI mode.
* @property {boolean} [ci] Disables interactive mode when executed in CI mode.
* @property {string} [repositoryPath] Relative path to the git repository.
* @property {SemVer} [semver] The selected semantic versioning. Defaults to `patch`.
*/

/**
Expand Down Expand Up @@ -331,11 +333,18 @@ async function publishPackagesToNpm( {
cwd: gitWorkingDirectoryPath,
} );

log( '>> Current npm user:' );
await command( 'npm whoami', {
cwd: gitWorkingDirectoryPath,
stdio: 'inherit',
} );

const beforeCommitHash = await git.getLastCommitHash(
gitWorkingDirectoryPath
);

const yesFlag = interactive ? '' : '--yes';
const noVerifyAccessFlag = interactive ? '' : '--no-verify-access';
if ( releaseType === 'next' ) {
log(
'>> Bumping version of public packages changed since the last release.'
Expand All @@ -351,7 +360,7 @@ async function publishPackagesToNpm( {

log( '>> Publishing modified packages to npm.' );
await command(
`npx lerna publish from-package --dist-tag next ${ yesFlag }`,
`npx lerna publish from-package --dist-tag next ${ yesFlag } ${ noVerifyAccessFlag }`,
{
cwd: gitWorkingDirectoryPath,
stdio: 'inherit',
Expand All @@ -360,7 +369,7 @@ async function publishPackagesToNpm( {
} else if ( releaseType === 'bugfix' ) {
log( '>> Publishing modified packages to npm.' );
await command(
`npx lerna publish ${ minimumVersionBump } --no-private ${ yesFlag }`,
`npx lerna publish ${ minimumVersionBump } --no-private ${ yesFlag } ${ noVerifyAccessFlag }`,
{
cwd: gitWorkingDirectoryPath,
stdio: 'inherit',
Expand All @@ -379,10 +388,13 @@ async function publishPackagesToNpm( {
);

log( '>> Publishing modified packages to npm.' );
await command( `npx lerna publish from-package ${ yesFlag }`, {
cwd: gitWorkingDirectoryPath,
stdio: 'inherit',
} );
await command(
`npx lerna publish from-package ${ yesFlag } ${ noVerifyAccessFlag }`,
{
cwd: gitWorkingDirectoryPath,
stdio: 'inherit',
}
);
}

const afterCommitHash = await git.getLastCommitHash(
Expand Down Expand Up @@ -458,17 +470,23 @@ async function runPackagesRelease( config, customMessages ) {
await askForConfirmation( 'Ready to go?' );
}

// Cloning the Git repository.
config.gitWorkingDirectoryPath = await runGitRepositoryCloneStep(
config.abortMessage
);
const temporaryFolders = [ config.gitWorkingDirectoryPath ];
const temporaryFolders = [];
if ( ! config.gitWorkingDirectoryPath ) {
// Cloning the Git repository.
config.gitWorkingDirectoryPath = await runGitRepositoryCloneStep(
config.abortMessage
);
temporaryFolders.push( config.gitWorkingDirectoryPath );
}

let pluginReleaseBranch;
if ( [ 'latest', 'next' ].includes( config.releaseType ) ) {
pluginReleaseBranch = await findPluginReleaseBranchName(
config.gitWorkingDirectoryPath
);
pluginReleaseBranch =
config.releaseType === 'next'
? 'trunk'
: await findPluginReleaseBranchName(
config.gitWorkingDirectoryPath
);
await runNpmReleaseBranchSyncStep( pluginReleaseBranch, config );
} else {
await checkoutNpmReleaseBranch( config );
Expand Down Expand Up @@ -510,10 +528,11 @@ async function runPackagesRelease( config, customMessages ) {
*
* @return {WPPackagesConfig} The config object.
*/
function getConfig( releaseType, { ci, semver } ) {
function getConfig( releaseType, { ci, repositoryPath, semver } ) {
return {
abortMessage: 'Aborting!',
gitWorkingDirectoryPath: process.cwd(),
gitWorkingDirectoryPath:
repositoryPath && join( process.cwd(), repositoryPath ),
interactive: ! ci,
minimumVersionBump: semver,
npmReleaseBranch: releaseType === 'next' ? 'wp/next' : 'wp/trunk',
Expand Down
34 changes: 15 additions & 19 deletions docs/contributors/code/release.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,35 +192,31 @@ The Gutenberg repository mirrors the [WordPress SVN repository](https://make.wor

- The `wp/trunk` branch contains the same version of packages published to npm with the `latest` distribution tag. The WordPress core consumes those packages directly in the `trunk` branch and uses them for public releases.
- The `wp/next` branch contains the same version of packages published to npm with the `next` distribution tag. Projects should use those packages for development purposes only.
- A Gutenberg branch targeting a specific WordPress major release (including its further minor increments) is created (example `wp/5.2`) based on the `wp/trunk` Gutenberg branch when the corresponding WordPress release branch is created. (This usually happens when the first `RC` of the next WordPress major version is released).
- A Gutenberg branch targeting a specific WordPress major release (including its further minor increments) is created (example `wp/5.2`) based on the `wp/trunk` Gutenberg branch when the corresponding WordPress release branch is created. (This usually happens when the `beta` or `RC` of the next WordPress major version is released).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean the WP release branch will be created at Beta going forward, or are we planning to create the Gutenberg release branch before the WP one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is strictly RC in WordPress core. I see it's scheduled for WP 6.0 at the time the first RC happens. In practice, it means we will have to cut a new wp/6.0 branch at the time when cherry-pick first commits from the Gutenberg plugin release that doesn't get included in the WordPress release. So that means a bit earlier.

I don't think it doesn't change much in the process, because previously we would use wp/trunk for publishing dedicated for WordPress core, and here we just use a different branch wp/6.0.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that makes sense. I think in WP too it would be better to create the release branch straight after Beta, but that might not change anytime soon, and nothing stops us from creating the Gutenberg release branch earlier.

To make the docs clear, perhaps we can specify that the Gutenberg release branch is created as soon as code freeze (or Beta 1, if there's no prior code freeze) happens in Core.


Release types and their schedule:

- [Synchronizing WordPress Trunk](#synchronizing-wordpress-trunk) (`latest` dist tag) – when there is no "feature-freeze" mode in WordPress Core, publishing happens every two weeks based on the newly created RC1 version of the Gutenberg plugin. Otherwise, only bug fixes get manually included and published to npm before every next beta and RC version of the following WordPress release.
- [Minor WordPress Releases](#minor-wordpress-releases) (`patch` dist tag) – only when bug fixes or security releases need to be backported into WordPress Core.
- [Development Releases](#development-releases) (`next` dist tag) – when there is a "feature-freeze" mode in WordPress Core, publishing can still happen every two weeks based on the new RC1 version of the Gutenberg plugin. It is also possible to perform development releases at any time when there is a need to test the upcoming changes.
- [Synchronizing Gutenberg Plugin](#synchronizing-gutenberg-plugin) (`latest` dist tag) – publishing happens every two weeks based on the newly created RC1 version of the Gutenberg plugin.
- [WordPress Releases](#wordpress-releases) (`patch` dist tag) – only when bug or security fixes need to be backported into WordPress Core.
- [Development Releases](#development-releases) (`next` dist tag) – it is also possible to perform development releases at any time when there is a need to test the upcoming changes.

There is also an option to perform [Standalone Bugfix Package Releases](#standalone-bugfix-package-releases) at will. It should be reserved only for critical bug fixes or security releases that must be published to _npm_ outside of a regular WordPress release cycle.
There is also an option to perform [Standalone Bugfix Package Releases](#standalone-bugfix-package-releases) at will. It should be reserved only for critical bug fixes or security releases that must be published to _npm_ outside of regular cycles.

### Synchronizing WordPress Trunk
### Synchronizing Gutenberg Plugin

For each Gutenberg plugin release, WordPress trunk should be synchronized.
For each Gutenberg plugin release, we also publish to npm an updated version of WordPress packages. This is automated with the [Release Tool](#release-tool) that handles releases for the Gutenberg plugin.

Note that the WordPress `trunk` branch can be closed or in "feature-freeze" mode. Usually, feature freeze in WordPress Core happens about 2 weeks before Beta 1 and remains in effect until RC1 when the `trunk` gets branched. During this period, the Gutenberg plugin releases should not be synchronized with WordPress Core.
We deliberately update the `wp/trunk` branch within the Gutenberg repo with the content from the Gutenberg release `release/*` (example `release/12.7`) branch at the time of the Gutenberg RC1 release. This is done to ensure that the `wp/trunk` branch is as close as possible to the latest version of the Gutenberg plugin. It also practically removes the chances of conflicts while backporting to `trunk` commits with updates applied during publishing to `package.json` and `CHANGELOG.md` files. In the past, we had many issues in that aspect when doing npm publishing after the regular Gutenberg release a week later. When publishing the new package versions to npm, we pick at least the `minor` version bump to account for future bugfix or security releases.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we update Core trunk after each Gutenberg release too? If not, does it really make sense to keep calling this branch wp/trunk if it's not in sync with Core trunk?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question. In the past, we used that branch several times to update the trunk in the WordPress core with changes published from wp/trunk before the beta for the major WordPress release. Nothing stops us from doing it on regular basis 😅

There are only 3 weeks between beta 1 (12 April) and RC 1 (3 May) for WP 6.0 so it's a relatively short period of time where wp/6.0 would exist in Gutenberg without 6.0 branch in WP core. Anyway, we can just rename it to wp/latest to better express its intent.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing stops us from doing it on regular basis 😅

Totally agree we should do this! I think the only thing that stops us is not wanting to throw all the super experimental/unstable stuff into Core, but we can maybe make more use of feature flags.

we can just rename it to wp/latest

That would be be ideal if it's not always going to match WP trunk 🙂


A different person usually synchronizes the WordPress `trunk` branch and publishes the npm packages. Therefore, you typically shouldn't need to worry about handling this for the normal plugin release process. However, if you are still unsure, ask in [the #core-editor Slack channel](https://wordpress.slack.com/archives/C02QB2JS7).

The process has three steps: 1) update the `wp/trunk` branch within the Gutenberg repo 2) publish the new package versions to npm 3) update the WordPress `trunk` branch.

All steps are automated via `./bin/plugin/cli.js npm-latest` command. You only have to run the command, but, for the record, the manual process would look very close to the following steps:
All steps are automated via `./bin/plugin/cli.js npm-latest` command. For the record, the manual process would look very close to the following steps:

1. Ensure the WordPress `trunk` branch is open for enhancements.
2. Get the last published Gutenberg release branch with `git fetch`.
3. Check out the `wp/trunk` branch.
4. Remove all files from the current branch: `git rm -r .`.
5. Check out all the files from the release branch: `git checkout release/x.x -- .`.
6. Commit all changes to the `wp/trunk` branch with `git commit -m "Merge changes published in the Gutenberg plugin vX.X release"` and push to the repository.
7. Update the `CHANGELOG.md` files of the packages with the new publish version calculated and commit to the `wp/trunk` branch. Assuming the package versions are written using this format `major.minor.patch`, make sure to bump at least the `minor` version number after every major WordPress release. For example, if the CHANGELOG of the package to be released indicates that the next unreleased version is `5.6.1`, choose `5.7.0` as a version in case of `minor` version. This is important as the patch version numbers should be reserved in case bug fixes are needed for a minor WordPress release (see below).
7. Update the `CHANGELOG.md` files of the packages with the new publish version calculated and commit to the `wp/trunk` branch. Assuming the package versions are written using this format `major.minor.patch`, make sure to bump at least the `minor` version bumps gets applied. For example, if the CHANGELOG of the package to be released indicates that the next unreleased version is `5.6.1`, choose `5.7.0` as a version in case of `minor` version. This is important as the patch version numbers should be reserved in case bug fixes are needed for a minor WordPress release (see below).
8. Log-in to npm via the console: `npm login`. Note that you should have 2FA enabled.
9. From the `wp/trunk` branch, install npm dependencies with `npm ci`.
10. Run the script `npm run publish:latest`.
Expand All @@ -229,11 +225,11 @@ All steps are automated via `./bin/plugin/cli.js npm-latest` command. You only h
- If the publishing process ends up incomplete (perhaps because it timed-out or an bad OTP was introduce) you can resume it via [`lerna publish from-package`](https://github.com/lerna/lerna/tree/HEAD/commands/publish#bump-from-package).
11. Finally, now that the npm packages are published, cherry-pick the commits created by lerna ("Publish" and the CHANGELOG update) into the `trunk` branch of Gutenberg.

### Minor WordPress Releases
### WordPress Releases

The following workflow is needed when bug fixes or security releases need to be backported into WordPress Core. This can happen in a few use-cases:
The following workflow is needed when bug or security fixes need to be backported into WordPress Core. This can happen in a few use-cases:

- During the `RC` period of the WordPress release cycle when `wp/X.Y` (example `wp/5.7`) branch for the release is already present.
- During the `beta` and `RC` periods of the WordPress release cycle when `wp/X.Y` (example `wp/5.7`) branch for the release is already present.
- For WordPress minor releases and WordPress security releases (example `5.1.1`).

1. Check out the relevant WordPress major branch (If the minor release is 5.2.1, check out `wp/5.2`).
Expand All @@ -256,7 +252,7 @@ Now, the npm packages should be ready and a patch can be created and committed i

### Standalone Bugfix Package Releases

The following workflow is needed when packages require bug fixes or security releases to be published to _npm_ outside of a regular WordPress release cycle.
The following workflow is needed when packages require bug fixes or security releases to be published to _npm_ outside of a regular release cycle.

Note: Both the `trunk` and `wp/trunk` branches are restricted and can only be _pushed_ to by the Gutenberg Core team.

Expand Down Expand Up @@ -313,7 +309,7 @@ The good news is that the rest of the process is automated with `./bin/plugin/cl

### Development Releases

As noted in the [Synchronizing WordPress Trunk](#synchronizing-wordpress-trunk) section, the WordPress trunk branch can be closed or in "feature-freeze" mode. Usually, this happens during the WordPress ongoing release cycle, which takes several weeks. It means that packages don't get any enhancements and new features during the ongoing WordPress major release process. Another type of release is available to address the limitation mentioned earlier and unblock ongoing development for projects that depend on WordPress packages. We are taking advantage of [package distribution tags](https://docs.npmjs.com/cli/v7/commands/npm-dist-tag) that make it possible to consume the future version of the codebase according to npm guidelines:
As noted in the [Synchronizing Gutenberg Plugin](#synchronizing-gutenberg-plugin) section, packages publishing happens every two weeks from the `wp/trunk` branch. It's also possible to use the development release to test the upcoming changes present in the `trunk` branch at any time. We are taking advantage of [package distribution tags](https://docs.npmjs.com/cli/v7/commands/npm-dist-tag) that make it possible to consume the future version of the codebase according to npm guidelines:

> By default, the `latest` tag is used by npm to identify the current version of a package, and `npm install <pkg>` (without any `@<version>` or `@<tag>` specifier) installs the `latest` tag. Typically, projects only use the `latest` tag for stable release versions, and use other tags for unstable versions such as prereleases.

Expand Down