Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Refactor scripts #60

Merged
merged 32 commits into from
Feb 14, 2025
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
32034fc
[wip]: Add new scripts
febo Feb 6, 2025
6be0456
[wip]: Use matric strategy
febo Feb 6, 2025
9455e7d
[wip]: Add setup action
febo Feb 6, 2025
6492a14
[wip]: Tweak output
febo Feb 6, 2025
7d2e20c
[wip]: Debug output
febo Feb 6, 2025
db04746
[wip]: Fix members parsing
febo Feb 6, 2025
32e6669
[wip]: Fix matrix strategy
febo Feb 6, 2025
9542414
[wip]: All process steps
febo Feb 6, 2025
d7c6e2a
[wip]: Add toolchains
febo Feb 6, 2025
ec01d10
[wip]: Add CI env variables
febo Feb 6, 2025
4f7ad78
[wip]: Remove nothrow
febo Feb 6, 2025
4bf599e
[wip]: Filter changes
febo Feb 6, 2025
e64f87f
[wip]: Add audit step
febo Feb 6, 2025
d376d52
[wip]: Update names
febo Feb 6, 2025
62d045b
[wip]: Add semver checks
febo Feb 7, 2025
233b5de
[wip]: Add publish workflow
febo Feb 7, 2025
a1de716
[wip]: Debug semver check
febo Feb 7, 2025
bc3cc18
[wip]: Refactor publish workflow
febo Feb 7, 2025
3bf2d9f
[wip]: Tweaks
febo Feb 7, 2025
0bc005f
[wip]: Fix set version
febo Feb 7, 2025
ab956d7
[wip]: Install cargo-edit
febo Feb 7, 2025
84b19e2
[wip]: Refactor
febo Feb 7, 2025
4693995
[wip]: Fix ci variables
febo Feb 7, 2025
343d9d3
[wip]: Fix import
febo Feb 7, 2025
503f35b
[wip]: Install solana
febo Feb 7, 2025
5982943
[wip]: Fix commands
febo Feb 7, 2025
7db5221
Fix formatting
febo Feb 7, 2025
45cecf0
Remove detect changes step
febo Feb 8, 2025
3a7dc55
Relax clippy check
febo Feb 8, 2025
ffa73d6
Review comments
febo Feb 14, 2025
4c846a0
Add crate/version output
febo Feb 14, 2025
23f37c2
Simplify range
febo Feb 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: Setup environment

inputs:
cargo-cache-key:
description: The key to cache cargo dependencies. Skips cargo caching if not provided.
required: false
toolchain:
description: Rust toolchain to install. Comma-separated string of [`build`, `format`, `lint`, `test`].
required: false
components:
description: Cargo components to install. Comma-separated string of [`audit`, `hack``, `release`, `semver-checks].
required: false
solana:
description: Install Solana if `true`. Defaults to `false`.
required: false

runs:
using: 'composite'
steps:
- name: Setup pnpm
uses: pnpm/action-setup@v3

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'

- name: Install Dependencies
run: pnpm install --frozen-lockfile
shell: bash

- name: Set Environment Variables
shell: bash
run: pnpm tsx ./scripts/setup/ci.mts

- name: Install Rust 'build' Toolchain
if: ${{ contains(inputs.toolchain, 'build') }}
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ env.TOOLCHAIN_BUILD }}

- name: Install Rust 'format' Toolchain
if: ${{ contains(inputs.toolchain, 'format') }}
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ env.TOOLCHAIN_FORMAT }}
components: rustfmt

- name: Install Rust 'lint' Toolchain
if: ${{ contains(inputs.toolchain, 'lint') }}
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ env.TOOLCHAIN_LINT }}
components: clippy

- name: Install Rust 'test' Toolchain
if: ${{ contains(inputs.toolchain, 'test') }}
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ env.TOOLCHAIN_TEST }}

- name: Install Solana
if: ${{ inputs.solana == 'true' }}
uses: solana-program/actions/install-solana@v1
with:
version: ${{ env.SOLANA_VERSION }}
cache: true

- name: Install 'cargo-audit'
if: ${{ contains(inputs.components, 'audit') }}
shell: bash
run: cargo install cargo-audit

- name: Install 'cargo-hack'
if: ${{ contains(inputs.components, 'hack') }}
shell: bash
run: cargo install cargo-hack

- name: Install 'cargo-release'
if: ${{ contains(inputs.components, 'release') }}
shell: bash
run: cargo install cargo-release

- name: Install 'cargo-semver-checks'
if: ${{ contains(inputs.components, 'semver-checks') }}
shell: bash
run: cargo install cargo-semver-checks

- name: Cache Cargo Dependencies
if: ${{ inputs.cargo-cache-key && !inputs.cargo-cache-fallback-key }}
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-${{ inputs.cargo-cache-key }}-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-${{ inputs.cargo-cache-key }}
106 changes: 59 additions & 47 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,65 +6,77 @@ on:
pull_request:

env:
RUST_VERSION: 1.78.0
SOLANA_VERSION: 1.18.20
CARGO_CACHE: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
CACHE: true

jobs:
lint:
name: Lint
audit:
name: Audit Dependencies
runs-on: ubuntu-latest
steps:
- name: Git checkout
uses: actions/checkout@v4
- name: Install components
uses: dtolnay/rust-toolchain@master

- name: Setup Environment
uses: ./.github/actions/setup
with:
components: clippy, rustfmt
toolchain: ${{ env.RUST_VERSION }}
- name: Formatting
run: cargo fmt --all --check
- name: Clippy
run: cargo clippy --all-targets --all-features --no-deps

build:
name: Build
needs: lint
cargo-cache-key: cargo-audit
components: audit

- name: cargo-audit
run: pnpm audit

filter:
name: Filter Workspace
runs-on: ubuntu-latest
needs: audit
outputs:
members: ${{ steps.filter.outputs.members }}
steps:
- name: Git checkout
- name: Git Checkout
uses: actions/checkout@v4
- name: Install Solana
uses: nifty-oss/actions/install-solana@v1
with:
version: ${{ env.SOLANA_VERSION }}
cache: true
- name: Cache cargo dependencies
uses: actions/cache@v4

- name: Setup Environment
uses: ./.github/actions/setup
with:
path: ${{ env.CARGO_CACHE }}
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo
- name: Build
run: cargo build --all-targets --all-features

test:
name: Test
needs: lint
cargo-cache-key: cargo-filter-workspace

- name: Filter
id: filter
run: pnpm tsx ./scripts/setup/members.mts

process:
name: Crate
needs: filter
runs-on: ubuntu-latest
strategy:
matrix:
member: ${{ fromJson(needs.filter.outputs.members) }}
steps:
- name: Git checkout
- name: Git Checkout
uses: actions/checkout@v4
- name: Cache cargo dependencies
uses: actions/cache@v4

- name: Setup Environment
uses: ./.github/actions/setup
with:
path: ${{ env.CARGO_CACHE }}
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo
- name: Build
run: cargo test --all-features
cargo-cache-key: cargo-${{ matrix.member }}
toolchain: build, format, lint, test
components: hack
solana: true

- name: fmt
run: pnpm format ${{ matrix.member }}

- name: clippy
run: pnpm clippy ${{ matrix.member }}

- name: cargo-doc
run: pnpm doc ${{ matrix.member }}

- name: cargo-hack
run: pnpm hack ${{ matrix.member }}

- name: build-sbf
run: pnpm build-sbf ${{ matrix.member }}

- name: test
run: pnpm test ${{ matrix.member }}
83 changes: 64 additions & 19 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,53 +6,98 @@ on:
crate:
description: Crate
required: true
default: pinocchio
default: sdk/pinocchio
type: choice
options:
- pinocchio
- pubkey
- programs/associated-token-account
- programs/system
- programs/token
- sdk/log/crate
- sdk/log/macro
- sdk/pinocchio
- sdk/pubkey
level:
description: Level
required: true
default: patch
type: choice
options:
- patch
- minor
- major
dry_run:
description: Dry run
required: true
default: true
type: boolean
create_release:
description: Create a GitHub release
required: true
type: boolean
default: true

env:
RUST_VERSION: 1.78.0
SOLANA_VERSION: 1.18.20
CARGO_CACHE: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
CACHE: true

jobs:
publish_release:
name: Publish
runs-on: ubuntu-latest
steps:
- name: Ensure CARGO_REGISTRY_TOKEN variable is set
env:
token: ${{ secrets.CARGO_REGISTRY_TOKEN }}
if: ${{ env.token == '' }}
run: |
echo "The CARGO_REGISTRY_TOKEN secret variable is not set"
echo "Go to \"Settings\" -> \"Secrets and variables\" -> \"Actions\" -> \"New repository secret\"."
exit 1

- name: Git checkout
uses: actions/checkout@v4

- name: Install Rust
uses: dtolnay/rust-toolchain@master
- name: Setup Environment
uses: ./.github/actions/setup
with:
toolchain: stable
cargo-cache-key: cargo-publish
toolchain: test
components: semver-checks
solana: true

- name: Build
run: pnpm build-sbf ${{ inputs.crate }}

- name: Test
run: pnpm test ${{ inputs.crate }}

- name: Set Git Author
run: |
git config --global user.email "[email protected]"
git config --global user.name "github-actions"

- name: Check semver
uses: obi1kenobi/cargo-semver-checks-action@v2
run: |
pnpm semver ${{ inputs.crate }} --release-type ${{ inputs.level }}

- name: Publish crate
- name: Publish Crate
id: publish
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
run: |
MANIFEST="./sdk/${{ inputs.crate }}/Cargo.toml"

if [ "${{ inputs.dry_run }}" == "true" ]; then
OPTIONS="--dry-run"
else
OPTIONS=""
fi

cargo publish --manifest-path $MANIFEST $OPTIONS
pnpm tsx ./scripts/publish.mts ${{ inputs.crate }} ${{ inputs.level }} $OPTIONS

- name: Push Commit and Tag
if: github.event.inputs.dry_run != 'true'
run: git push origin --follow-tags

- name: Create GitHub release
if: github.event.inputs.dry_run != 'true' && github.event.inputs.create_release == 'true'
uses: ncipollo/release-action@v1
with:
tag: ${{ steps.publish.outputs.crate }}@v${{ steps.publish.outputs.version }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
/node_modules
/target
9 changes: 9 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"semi": true,
"singleQuote": true,
"trailingComma": "es5",
"useTabs": false,
"tabWidth": 2,
"arrowParens": "always",
"printWidth": 80
}
13 changes: 11 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,14 @@ repository = "https://github.com/anza-xyz/pinocchio"

[workspace.dependencies]
five8_const = "0.1.3"
pinocchio = { path = "sdk/pinocchio", version = ">= 0.6, <= 0.7" }
pinocchio-pubkey = { path = "sdk/pubkey", version = "0.2.1" }
pinocchio = { path = "sdk/pinocchio", version = ">= 0.6" }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the version specified like that?

Copy link
Collaborator Author

@febo febo Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably me not using the workspace dependencies correctly. I was trying to avoid bumping dependencies on other packages unnecessarily. For example, if pinocchio-pubkey can work with multiple versions of pinocchio and pinocchio has been bumped to 0.7, I still want to specify that pinocchio-pubkey works with anything from >= 0.6.

But perhaps this is not the correct way to do that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, let's just keep them all at the same version for now, I think it'll make things simpler in the long run.

pinocchio-pubkey = { path = "sdk/pubkey", version = "0.2" }

[workspace.metadata.cli]
solana = "2.1.0"

[workspace.metadata.toolchains]
build = "1.81.0"
format = "nightly-2024-08-08"
lint = "nightly-2024-08-08"
test = "1.81.0"
24 changes: 24 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"private": true,
"scripts": {
"audit": "tsx ./scripts/audit.mjs",
"build-sbf": "tsx ./scripts/build-sbf.mjs",
"clippy": "tsx ./scripts/clippy.mjs",
"doc": "tsx ./scripts/doc.mjs",
"format": "tsx ./scripts/format.mjs",
"hack": "tsx ./scripts/hack.mjs",
"lint": "tsx ./scripts/lint.mjs",
"semver": "tsx ./scripts/semver.mjs",
"test": "tsx ./scripts/test.mjs"
},
"devDependencies": {
"@iarna/toml": "^2.2.5",
"tsx": "^4.19.2",
"typescript": "^5.5.2",
"zx": "^7.2.3"
},
"engines": {
"node": ">=v20.0.0"
},
"packageManager": "[email protected]"
}
Loading
Loading