Build a new release process for this repository (#1416)

Currently the release process of this repository is ad-hoc. Someone
requests it, I guess what crates need a major/minor bump, I make a PR,
and then I publish. No one else can really make a release and I myself
make lots of mistakes. The goal of this commit is to design a new
process that is less error-prone and can be driven by someone other than
me.

Lots of bits and pieces here are lifted from Wasmtime's own release
process, but the general idea here is:

* Most crates now all share the same version number of 0.X.0. Exceptions
  include:

  * `wat` - uses 1.X.0
  * `wast` - uses X.0.0
  * `wasm-tools` - uses 1.X.0

  These exceptions are required for previously published versions.
  Notably `wat` and `wasm-tools` are "API stable" so are known to not
  have breaking changes anywhere on the horizon. The `wast` crate was
  already published at 1.0.0+ so can't use the 0.X.0 numbering of other
  crates.

* The initial version number checked in here, the X in 0.X.0, is 199.
  This means that the first published version with this new scheme will
  be 200, explained below.

* To simplify CI and releases merge queues are now enabled for this
  repository.

* Version bumps are now manually requested in the "Actions" tab of the
  repository. The "Automated Release Process" workflow is triggered
  manually with either `release-minor` or `release-major` as an
  argument. When run this will execute a script to update version
  numbers and a PR is posted.

* Upon merging the automatically produced PR, which has a "magical
  string" in the commit messages, a release is created when the PR
  merges to `main`. This uses artifacts previously built to upload to
  GitHub Releases. This additionally will publish all crates.

* Bumping version numbers is slightly different than before. For crates
  using `0.X.Y`, they're bumped to `0.(X+1).Y` or `0.X.(Y+1)` depending
  on the release. For `X.0.Y` crates it's either `(X+1).0.Y` or
  `X.0.(Y+1)`. For `1.X.Y` crates it's either `1.(X+1).Y` or
  `1.X.(Y+1)`. This enables having a patch release for any historical
  release.

* Release branches are not made yet. Unlike Wasmtime releases are made
  by merging a PR to `main`, not a PR to a `release-*` branch. Only a
  tag is created (via CI). Patch releases will require manually creating
  a branch first. I also haven't tested patch releases due to how rare
  they are.

* This repository will no longer create a tag-per-crate-and-version
  pair. Instead only a single tag will be created of the form `v1.X.Y`
  depending on the version number of the `wasm-tools` CLI itself.

The goal of the versioning scheme is to be able to understand where
everything is versioned at at-a-glance. There's a clear correlation
between `wasmparser` and `wasm-smith` now, for example. Additionally
I've opted to start everything at 199 since the first released to be
published will be one greater which will be 200. I've chosen 200 because
`wasmparser` is already at 121 and I like starting on "nice round
numbers".

I've attempted to leave documentation for all the various places,
starting in `README.md` about the high-level steps necessary to make a
release. My plan is to make a release immediately after this merges to
do a run-through of the whole process to make sure it all works.
Assuming everything goes smoothly I'll hope to get someone else to
make the next release to ensure that goes smoothly as well. Assuming
that goes well we next just hope that folks don't get too angry about
the versioning scheme selected.

.github/workflows/main.yml

@@ -1,17 +1,56 @@
 name: CI
 on:
-  push:
-    branches: [main]
   pull_request:
-    branches: [main]
+  merge_group:
 
 # Cancel any in-flight jobs for the same PR/branch so there's only one active
 # at a time
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
+defaults:
+  run:
+    shell: bash
+
 jobs:
+  build:
+    name: Build wasm-tools
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        include:
+        - build: x86_64-linux
+          os: ubuntu-latest
+        - build: x86_64-macos
+          os: macos-latest
+        - build: aarch64-macos
+          os: macos-latest
+          target: aarch64-apple-darwin
+        - build: x86_64-windows
+          os: windows-latest
+        - build: aarch64-linux
+          os: ubuntu-latest
+          target: aarch64-unknown-linux-gnu
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        submodules: true
+    - run: rustup update stable --no-self-update && rustup default stable
+    - uses: bytecodealliance/wasmtime/.github/actions/[email protected]
+      with:
+        name: ${{ matrix.build }}
+    - run: |
+        echo CARGO_BUILD_TARGET=${{ matrix.target }} >> $GITHUB_ENV
+        rustup target add ${{ matrix.target }}
+      if: matrix.target != ''
+    - run: $CENTOS cargo build --release
+    - run: ./ci/build-tarballs.sh "${{ matrix.build }}" "${{ matrix.target }}"
+    - uses: actions/upload-artifact@v3
+      with:
+        name: bins-${{ matrix.build }}
+        path: dist
+
   test:
     name: Test
     runs-on: ${{ matrix.os }}
@@ -35,26 +74,25 @@ jobs:
             os: windows-latest
             rust: stable
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
       with:
         submodules: true
     - name: Install Rust (rustup)
       run: rustup update ${{ matrix.rust }} --no-self-update && rustup default ${{ matrix.rust }}
-      shell: bash
     - run: cargo test --locked --all
     - run: cargo test --locked -p wasmparser --benches
     - run: cargo test --locked -p wasm-encoder --all-features
     - run: cargo test -p wasm-smith --features wasmparser
     - run: cargo build --manifest-path crates/wast/Cargo.toml --no-default-features
     - run: cargo build --manifest-path crates/wast/Cargo.toml --no-default-features --features wasm-module
-    - run: cmake -S ${{github.workspace}}/examples -B ${{github.workspace}}/examples/build -DCMAKE_BUILD_TYPE=Release
-    - run: cmake --build ${{github.workspace}}/examples/build --config Release
+    - run: cmake -S examples -B examples/build -DCMAKE_BUILD_TYPE=Release
+    - run: cmake --build examples/build --config Release
 
   wasm:
     name: Test on WebAssembly
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
       with:
         submodules: true
     - name: Install Rust
@@ -77,7 +115,7 @@ jobs:
     name: Rustfmt
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Install Rust
       run: rustup update stable && rustup default stable && rustup component add rustfmt
     # Note that this doesn't use `cargo fmt` because that doesn't format
@@ -89,7 +127,7 @@ jobs:
     name: Fuzz
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
       with:
         submodules: true
     - name: Install Rust
@@ -101,7 +139,7 @@ jobs:
   check:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - run: cargo check --benches -p wasm-smith
       - run: cargo check --no-default-features
       - run: cargo check --no-default-features --features print
@@ -133,5 +171,45 @@ jobs:
   doc:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - run: cargo doc --all
+
+  verify-publish:
+    if: github.repository_owner == 'bytecodealliance'
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        submodules: true
+    - run: rustup update stable && rustup default stable
+    - run: rustc ci/publish.rs
+    # Make sure the tree is publish-able as-is
+    - run: ./publish verify
+    # Make sure we can bump version numbers for the next release
+    - run: ./publish bump
+
+  # "Join node" which the merge queue waits on.
+  ci-status:
+    name: Record the result of testing and building steps
+    runs-on: ubuntu-latest
+    needs:
+      - test
+      - wasm
+      - rustfmt
+      - fuzz
+      - check
+      - doc
+      - build
+      - verify-publish
+    if: always()
+
+    steps:
+    - name: Successful test and build
+      if: ${{ !(contains(needs.*.result, 'failure')) }}
+      run: exit 0
+    - name: Failing test and build
+      if: ${{ contains(needs.*.result, 'failure') }}
+      run: exit 1
+    - name: Report failure on cancellation
+      if: ${{ contains(needs.*.result, 'cancelled') || cancelled() }}
+      run: exit 1

.github/workflows/publish.yml

@@ -0,0 +1,87 @@
+# Publication half of the release process for this repository. This runs on
+# pushes to `main` and will detect a magical string in commit messages. When
+# found a tag will be created, pushed, and then everything is published.
+
+name: Publish Artifacts
+on:
+  push:
+    branches: [main]
+
+permissions:
+  contents: write
+
+jobs:
+  create_tag:
+    name: Publish artifacts of build
+    runs-on: ubuntu-latest
+    if: |
+      github.repository_owner == 'bytecodealliance'
+      && github.event_name == 'push'
+      && github.ref == 'refs/heads/main'
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        submodules: true
+        fetch-depth: 0
+
+    - run: rustup update stable && rustup default stable
+
+    # If this is a push to `main` see if the push has an indicator saying that
+    # a tag should be made. If so create one and push it.
+    - name: Test if tag is needed
+      run: |
+        git log ${{ github.event.before }}...${{ github.event.after }} | tee main.log
+        version=$(./ci/print-current-version.sh)
+        echo "version: $version"
+        echo "version=$version" >> $GITHUB_OUTPUT
+        echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
+        if grep -q "automatically-tag-and-release-this-commit" main.log; then
+          echo push-tag
+          echo "push_tag=yes" >> $GITHUB_OUTPUT
+        else
+          echo no-push-tag
+          echo "push_tag=no" >> $GITHUB_OUTPUT
+        fi
+      id: tag
+
+    - name: Push the tag
+      run: |
+        git_refs_url=$(jq .repository.git_refs_url $GITHUB_EVENT_PATH | tr -d '"' | sed 's/{\/sha}//g')
+        curl -iX POST $git_refs_url \
+          -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
+          -d @- << EOF
+        {
+          "ref": "refs/tags/v${{ steps.tag.outputs.version }}",
+          "sha": "${{ steps.tag.outputs.sha }}"
+        }
+        EOF
+      if: steps.tag.outputs.push_tag == 'yes'
+
+    - run: |
+        sha=${{ github.sha }}
+        run_id=$(
+          gh api -H 'Accept: application/vnd.github+json' \
+              /repos/${{ github.repository }}/actions/workflows/main.yml/runs\?exclude_pull_requests=true \
+              | jq '.workflow_runs' \
+              | jq "map(select(.head_commit.id == \"$sha\"))[0].id" \
+        )
+        gh run download $run_id
+        ls
+        find bins-*
+        mkdir dist
+        mv bins-*/* dist
+      env:
+        GH_TOKEN: ${{ github.token }}
+
+    - uses: softprops/action-gh-release@v1
+      if: steps.tag.outputs.push_tag == 'yes'
+      with:
+        files: "dist/*"
+        tag_name: v${{ steps.tag.outputs.version }}
+
+    - run: |
+        rustc ci/publish.rs
+        ./publish publish
+      env:
+        CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
+      if: steps.tag.outputs.push_tag == 'yes'

.github/workflows/release-process.yml

@@ -0,0 +1,70 @@
+# Initiation half of the release process for this repository.
+#
+# This is triggered manually through the github actions UI and will execute
+# `./publish bump` (or `bump-patch`). Afterwards the result will be pushed to a
+# branch in the main repository and a PR will be opened. This PR, when merged,
+# will trigger the second half in `publish.yml`.
+
+name: "Automated Release Process"
+on:
+  # Allow manually triggering this request via the button on the action
+  # workflow page.
+  workflow_dispatch:
+    inputs:
+      action:
+        description: 'Publish script argument: "bump", or "bump-patch"'
+        required: false
+        default: 'bump'
+
+jobs:
+  release_process:
+    name: Run the release process
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: true
+      - name: Setup
+        run: |
+          rustc ci/publish.rs
+          git config user.name 'Auto Release Process'
+          git config user.email '[email protected]'
+          git remote set-url origin https://git:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
+
+      - name: Bump version number
+        run: ./publish ${{ github.event.inputs.action }}
+
+      - name: Prep PR metadata
+        run: |
+          set -ex
+          git fetch origin
+
+          cur=$(./ci/print-current-version.sh)
+
+          git commit --allow-empty -a -F-<<EOF
+          Release ${{ github.event.repository.name }} $cur
+
+          [automatically-tag-and-release-this-commit]
+          EOF
+
+          # Push the result to a branch and setup metadata for the step below
+          # that creates a PR
+          git push origin HEAD:ci/release-$cur
+          echo "PR_HEAD=ci/release-$cur" >> $GITHUB_ENV
+          echo "PR_TITLE=Release ${{ github.event.repository.name }} $cur" >> $GITHUB_ENV
+          echo "PR_BASE=main" >> $GITHUB_ENV
+          cat > pr-body <<-EOF
+          This is an automated pull request from CI to release
+          ${{ github.event.repository.name }} $cur when merged. The commit
+          message for this PR has a marker that is detected by CI to create
+          tags and publish crate artifacts.
+
+          When first opened this PR will not have CI run because it is generated
+          by a bot. A maintainer should close this PR and then reopen it to
+          trigger CI to execute which will then enable merging this PR.
+          EOF
+
+      - name: Make a PR
+        run: gh pr create -B "$PR_BASE" -H "$PR_HEAD" --title "$PR_TITLE" --body "$(cat ./pr-body)"
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}