Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use folders instead of separate npm packages #70

Merged
merged 6 commits into from
Nov 15, 2022
Merged

Use folders instead of separate npm packages #70

merged 6 commits into from
Nov 15, 2022

Conversation

adamziel
Copy link
Collaborator

@adamziel adamziel commented Nov 15, 2022 •
edited
Loading

Description

This PR greatly simplifies the repository setup by getting rid of the separate package.json files as they made the setup unnecessarily complex.

Before this PR

  • Multiple tsconfig.json setups meant we needed to rebuild types in a particular order to even get the dev environment to work.
  • Esbuild needed two steps to first build each package separately and only then build the entire project. Worse, step one used ES Modules and step two used CommonJS. Using tsconfig.json made it extra hard. Substituting env variables caused conflicts.
  • Generating the documentation required 8 separate configuration files
  • Adding a new package was tedious. It required adding new package.json, tsconfig.json, api-documenter.json, adjusting all the build pipelines, ensuring the build steps are triggered in the correct order etc.

After this PR

  • There are two tsconfig.json files in the repo root and that's it. One is for the main-thread code and the other is for the worker code.
  • ESbuild runs in a single step and is configured with a single JS object.
  • There are no more documentation generator configs, it's all automated.
  • Introducing a new section of the repository is as simple as creating a new directory. There's no need to think about configs, build pipelines etc

@adamziel adamziel force-pushed the refactor/no-packages branch 2 times, most recently from d534592 to 96bacf9 Compare November 15, 2022 18:59
@adamziel adamziel changed the title Refactor/no packages Use folders instead of separate npm packages Nov 15, 2022
@adamziel adamziel self-assigned this Nov 15, 2022
@adamziel adamziel marked this pull request as ready for review November 15, 2022 19:24
@adamziel adamziel merged commit 44289e6 into trunk Nov 15, 2022
@adamziel adamziel deleted the refactor/no-packages branch November 15, 2022 19:25
@adamziel adamziel mentioned this pull request Mar 8, 2023
Closed
adamziel added a commit that referenced this pull request Mar 10, 2023
@adamziel
Migrate to Yarn workspaces
fd7e6ce 
This PR splits the repository into five node packages that are also yarn workspaces:

- php-cli
- php-wasm
- typescript-reference-doc-generator
- wordpress-playground
- wordpress-plugin-ide

This effectively undoes [Use folders instead of separate npm packages](#70), so you may ask "why bother?"

With this PR:

* The build process is straightforward. There's no `gulp` and there's no mountain of custom logic in `esbuild.js`. Files are moved around the repo using a simple `copy` plugin or an `import` statement. There's no custom cache busting logic – it's handled by the build tools instead
* `php-wasm` is now reusable and can be published in npm. This means it can be easily reused outside of WordPress Playground.
* Each package has its own build process that caters to its specific needs. For example, `php-cli` outputs a node.js script. Web workers can now be bundled as `iife` (which is the only way to use them in firefox) while other assets are es modules.
* There are less tasks in general. For example, TypeScript types are handled by rollup in a few packages.

At the same time, none of the [original downsides of using packages ](#70) are present:

* There's no global build pipeline. Every package outputs a complete build artifact that can be directly imported in case of libs, or used in the browser in case of the Playground website.
* Yarn workspaces ensure the tasks are executed in the correct order
* tsconfig.json and package.json can mostly be copied, there's no `api-documenter.json` at all.
adamziel added a commit that referenced this pull request Mar 10, 2023
@adamziel
Migrate to Yarn workspaces (#146)
a7601f0 
This PR splits the repository into five node packages that are also yarn workspaces:

- php-cli
- php-wasm
- typescript-reference-doc-generator
- wordpress-playground
- wordpress-plugin-ide

This effectively undoes [Use folders instead of separate npm packages](#70), so you may ask "why bother?"

With this PR:

* The build process is straightforward. There's no `gulp` and there's no mountain of custom logic in `esbuild.js`. Files are moved around the repo using a simple `copy` plugin or an `import` statement. There's no custom cache busting logic – it's handled by the build tools instead
* `php-wasm` is now reusable and can be published in npm. This means it can be easily reused outside of WordPress Playground.
* Each package has its own build process that caters to its specific needs. For example, `php-cli` outputs a node.js script. Web workers can now be bundled as `iife` (which is the only way to use them in firefox) while other assets are es modules.
* There are less tasks in general. For example, TypeScript types are handled by rollup in a few packages.

At the same time, none of the [original downsides of using packages ](#70) are present:

* There's no global build pipeline. Every package outputs a complete build artifact that can be directly imported in case of libs, or used in the browser in case of the Playground website.
* Yarn workspaces ensure the tasks are executed in the correct order
* tsconfig.json and package.json can mostly be copied, there's no `api-documenter.json` at all.
@adamziel adamziel mentioned this pull request Mar 16, 2023
Merged
adamziel added a commit that referenced this pull request Mar 16, 2023
@adamziel
Migrate to NX-based build (#151)
d676988 
## Description

Sets up the correct build pipeline for all parts of Playground and PHP.wasm. This enables a public release of the [Playground API](#149) npm package!

I've been [struggling](#146) with [this](#70) for [a while](#150) and couldn't understand what's so hard. NX made it apparent – look at this dependency graph!

<img width="1291" alt="CleanShot 2023-03-16 at 23 16 26@2x" src="https://user-images.githubusercontent.com/205419/225764795-7fa8e972-09f8-41ef-aac2-1c96bd100ea0.png">

No wonder it's been almost impossible to set everything up by hand!

## Usage

Start with `yarn install`

### Shortcuts

To start a copy of `wasm.wordpress.net` locally, run `yarn run dev`.
To build it, run `yarn run build`.

### Fully qualified commands

In reality, these `yarn run` commands are just triggering the underlying project's nx `dev` and `build` commands:

```bash
nx dev playground-website
nx build playground-website
```

Here's a few more interesting commands:

```bash
# Build and run PHP.wasm CLI
nx start php-wasm-cli

# Build latest WordPress releases
nx recompile-wordpress:all playground-remote

# Recompile PHP 5.6 - 8.2 releases to .wasm for web
nx recompile-php:all php-wasm-web

# Recompile PHP 5.6 - 8.2 releases to .wasm for node
nx recompile-php:all php-wasm-node

# Builds markdown files for readthedocs site
nx build docs-site

# Builds the Playground Client npm package
nx build playground-client
```

## NX is the tool Playground needed from the outset

It's ridiculous how many problems this solves:

* The build pipeline is explicitly defined and easy to modify
* Tasks only run once their dependencies are ready
* The dev mode works and is fast
* The build works and is fast
* We get CI checks to confirm the entire build process still works (which solves #150)
* Cross-package TypeScript just works
* There are linters and formatters (which solves #14)
* Documentation is correctly generated from the latest built artifacts
* There are nice generators for bootstraping new packages and moving the existing ones around
* There are checks to ensure the private `php-wasm-common` package is not imported by anything else than `php-wasm-web` and `php-wasm-node`

## Next steps

* Add Lerna to harness package publishing
* Additional developer documentation for the nx-based flow

Related to #148 and #147
Pookie717 added a commit to Pookie717/wordpress-playground that referenced this pull request Oct 1, 2023
@Pookie717
Migrate to Yarn workspaces
5470439 
This PR splits the repository into five node packages that are also yarn workspaces:

- php-cli
- php-wasm
- typescript-reference-doc-generator
- wordpress-playground
- wordpress-plugin-ide

This effectively undoes [Use folders instead of separate npm packages](WordPress/wordpress-playground#70), so you may ask "why bother?"

With this PR:

* The build process is straightforward. There's no `gulp` and there's no mountain of custom logic in `esbuild.js`. Files are moved around the repo using a simple `copy` plugin or an `import` statement. There's no custom cache busting logic – it's handled by the build tools instead
* `php-wasm` is now reusable and can be published in npm. This means it can be easily reused outside of WordPress Playground.
* Each package has its own build process that caters to its specific needs. For example, `php-cli` outputs a node.js script. Web workers can now be bundled as `iife` (which is the only way to use them in firefox) while other assets are es modules.
* There are less tasks in general. For example, TypeScript types are handled by rollup in a few packages.

At the same time, none of the [original downsides of using packages ](WordPress/wordpress-playground#70) are present:

* There's no global build pipeline. Every package outputs a complete build artifact that can be directly imported in case of libs, or used in the browser in case of the Playground website.
* Yarn workspaces ensure the tasks are executed in the correct order
* tsconfig.json and package.json can mostly be copied, there's no `api-documenter.json` at all.
Pookie717 added a commit to Pookie717/wordpress-playground that referenced this pull request Oct 1, 2023
@Pookie717
Migrate to Yarn workspaces (#146)
ec7d228 
This PR splits the repository into five node packages that are also yarn workspaces:

- php-cli
- php-wasm
- typescript-reference-doc-generator
- wordpress-playground
- wordpress-plugin-ide

This effectively undoes [Use folders instead of separate npm packages](WordPress/wordpress-playground#70), so you may ask "why bother?"

With this PR:

* The build process is straightforward. There's no `gulp` and there's no mountain of custom logic in `esbuild.js`. Files are moved around the repo using a simple `copy` plugin or an `import` statement. There's no custom cache busting logic – it's handled by the build tools instead
* `php-wasm` is now reusable and can be published in npm. This means it can be easily reused outside of WordPress Playground.
* Each package has its own build process that caters to its specific needs. For example, `php-cli` outputs a node.js script. Web workers can now be bundled as `iife` (which is the only way to use them in firefox) while other assets are es modules.
* There are less tasks in general. For example, TypeScript types are handled by rollup in a few packages.

At the same time, none of the [original downsides of using packages ](WordPress/wordpress-playground#70) are present:

* There's no global build pipeline. Every package outputs a complete build artifact that can be directly imported in case of libs, or used in the browser in case of the Playground website.
* Yarn workspaces ensure the tasks are executed in the correct order
* tsconfig.json and package.json can mostly be copied, there's no `api-documenter.json` at all.
Pookie717 added a commit to Pookie717/wordpress-playground that referenced this pull request Oct 1, 2023
@Pookie717
Migrate to NX-based build (#151)
57d3b59 
## Description

Sets up the correct build pipeline for all parts of Playground and PHP.wasm. This enables a public release of the [Playground API](WordPress/wordpress-playground#149) npm package!

I've been [struggling](WordPress/wordpress-playground#146) with [this](WordPress/wordpress-playground#70) for [a while](WordPress/wordpress-playground#150) and couldn't understand what's so hard. NX made it apparent – look at this dependency graph!

<img width="1291" alt="CleanShot 2023-03-16 at 23 16 26@2x" src="https://user-images.githubusercontent.com/205419/225764795-7fa8e972-09f8-41ef-aac2-1c96bd100ea0.png">

No wonder it's been almost impossible to set everything up by hand!

## Usage

Start with `yarn install`

### Shortcuts

To start a copy of `wasm.wordpress.net` locally, run `yarn run dev`.
To build it, run `yarn run build`.

### Fully qualified commands

In reality, these `yarn run` commands are just triggering the underlying project's nx `dev` and `build` commands:

```bash
nx dev playground-website
nx build playground-website
```

Here's a few more interesting commands:

```bash
# Build and run PHP.wasm CLI
nx start php-wasm-cli

# Build latest WordPress releases
nx recompile-wordpress:all playground-remote

# Recompile PHP 5.6 - 8.2 releases to .wasm for web
nx recompile-php:all php-wasm-web

# Recompile PHP 5.6 - 8.2 releases to .wasm for node
nx recompile-php:all php-wasm-node

# Builds markdown files for readthedocs site
nx build docs-site

# Builds the Playground Client npm package
nx build playground-client
```

## NX is the tool Playground needed from the outset

It's ridiculous how many problems this solves:

* The build pipeline is explicitly defined and easy to modify
* Tasks only run once their dependencies are ready
* The dev mode works and is fast
* The build works and is fast
* We get CI checks to confirm the entire build process still works (which solves #150)
* Cross-package TypeScript just works
* There are linters and formatters (which solves #14)
* Documentation is correctly generated from the latest built artifacts
* There are nice generators for bootstraping new packages and moving the existing ones around
* There are checks to ensure the private `php-wasm-common` package is not imported by anything else than `php-wasm-web` and `php-wasm-node`

## Next steps

* Add Lerna to harness package publishing
* Additional developer documentation for the nx-based flow

Related to WordPress/wordpress-playground#148 and WordPress/wordpress-playground#147
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@adamziel adamziel

Labels
[Type] Developer Experience
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@adamziel