Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI improvements #230

Merged
merged 55 commits into from
Feb 25, 2025
Merged
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
dfb49ca
LLVM release workflows
mutantcornholio Feb 17, 2025
850d042
test
AndWeHaveAPlan Feb 19, 2025
0cf5733
get-llvm
AndWeHaveAPlan Feb 19, 2025
1877963
path
AndWeHaveAPlan Feb 19, 2025
0c79bb1
checkout
AndWeHaveAPlan Feb 19, 2025
f53a2b5
steps
AndWeHaveAPlan Feb 19, 2025
bf129b0
run
AndWeHaveAPlan Feb 19, 2025
250ccaf
github
AndWeHaveAPlan Feb 19, 2025
be07392
log
AndWeHaveAPlan Feb 19, 2025
0e11750
id
AndWeHaveAPlan Feb 19, 2025
e032730
z
AndWeHaveAPlan Feb 19, 2025
7301568
strip-components=1
AndWeHaveAPlan Feb 19, 2025
4dfcf00
logs
AndWeHaveAPlan Feb 19, 2025
49b5945
log
AndWeHaveAPlan Feb 19, 2025
ad3173a
tests
AndWeHaveAPlan Feb 19, 2025
d6efab7
errors
AndWeHaveAPlan Feb 19, 2025
29d6e9e
test-parity-large
AndWeHaveAPlan Feb 19, 2025
9821002
LLVM release workflows
mutantcornholio Feb 17, 2025
70318da
Using released LLVM in revive build
mutantcornholio Feb 15, 2025
8cc9e8e
bump
mutantcornholio Feb 19, 2025
6d7792d
remove sort
AndWeHaveAPlan Feb 19, 2025
6120545
Merge branch 'main' into es/actions-test
AndWeHaveAPlan Feb 19, 2025
559df54
Using released LLVM in revive build
mutantcornholio Feb 15, 2025
f400709
bump
mutantcornholio Feb 19, 2025
3fdad7d
test build-revive-wasm
AndWeHaveAPlan Feb 19, 2025
7cf668b
push
AndWeHaveAPlan Feb 19, 2025
ffb40e1
Merge branch 'main' into es/actions-test
AndWeHaveAPlan Feb 19, 2025
0bfd064
log
AndWeHaveAPlan Feb 19, 2025
701ca1a
log
AndWeHaveAPlan Feb 19, 2025
779d55f
emsdk install
AndWeHaveAPlan Feb 19, 2025
db4b61c
action
AndWeHaveAPlan Feb 19, 2025
395212b
Using workflow_dispatch for LLVM releases
mutantcornholio Feb 20, 2025
52c90e3
Merge branch 'main' into es/actions-test
AndWeHaveAPlan Feb 20, 2025
5736eea
Merge branch 'main' into es/actions-test
AndWeHaveAPlan Feb 23, 2025
97dea83
get-emsdk
AndWeHaveAPlan Feb 23, 2025
e9762ad
emsdk version
AndWeHaveAPlan Feb 23, 2025
b56ea1c
release tests
AndWeHaveAPlan Feb 23, 2025
308ea92
clenup
AndWeHaveAPlan Feb 24, 2025
2f91427
Merge pull request #7 from paritytech/es/actions-test
AndWeHaveAPlan Feb 24, 2025
66e7270
rearrange create-release
AndWeHaveAPlan Feb 24, 2025
8b8203d
update release test
AndWeHaveAPlan Feb 24, 2025
ff6b6e6
no draft
AndWeHaveAPlan Feb 24, 2025
d9f66cc
target_commitish
AndWeHaveAPlan Feb 24, 2025
7016473
version 11
AndWeHaveAPlan Feb 24, 2025
8668fa4
cargo.lock versions
AndWeHaveAPlan Feb 24, 2025
67e4f34
remove wf tests
AndWeHaveAPlan Feb 24, 2025
cc0e609
description + typos
AndWeHaveAPlan Feb 24, 2025
5c9bf28
llvm-release concurrency
AndWeHaveAPlan Feb 24, 2025
bce9858
remove pull_request
AndWeHaveAPlan Feb 24, 2025
d9371ac
Merge branch 'main' into main
AndWeHaveAPlan Feb 24, 2025
2891483
delete script
AndWeHaveAPlan Feb 24, 2025
f8989d4
remove matrix.arch from job name
AndWeHaveAPlan Feb 24, 2025
7ba34fa
Update .github/workflows/release.yml
AndWeHaveAPlan Feb 24, 2025
981b075
Merge branch 'main' into main
xermicus Feb 25, 2025
fb13108
Apply suggestions from code review
AndWeHaveAPlan Feb 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/actions/get-emsdk/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: "get emsdk"
inputs:
version:
description: ""
required: false
default: "3.1.64"


runs:
using: "composite"
steps:

- name: install emsdk
shell: bash
run: |
git clone https://github.com/emscripten-core/emsdk.git ./emsdk/
cd emsdk
git checkout tags/${{ inputs.version }}
./emsdk install ${{ inputs.version }}
./emsdk activate ${{ inputs.version }}
91 changes: 91 additions & 0 deletions .github/actions/get-llvm/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# example:
#
# - name: get llvm
# uses: ./.github/actions/get-llvm
# with:
# releasePrefix: llvm-
# artifactArch: macos-arm64
# dir: target-llvm/macos

name: "get llvm"
inputs:
artifactArch:
required: true
releasePrefix:
description: "LLVM release tag prefix to search"
required: false
default: "llvm-"
dir:
description: "Archive extract path (`tar -C`)"
required: false
default: "./"
stripComponents:
description: "Strip UMBER leading components from file names on extraction (`tar --strip-components`)"
required: false
default: 0


runs:
using: "composite"
steps:
- name: find asset
id: find
uses: actions/github-script@v7
env:
releasePrefix: ${{ inputs.releasePrefix }}
artifactArch: ${{ inputs.artifactArch }}
with:
result-encoding: string
script: |
let page = 1;
let releases = [];

let releasePrefix = process.env.releasePrefix
let artifactArch = process.env.artifactArch

do {
const res = await github.rest.repos.listReleases({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 50,
page,
});

releases = res.data
releases.sort((a, b) => {
return (a.published_at < b.published_at) ? 1 : ((a.published_at > b.published_at) ? -1 : 0);
});

let llvmLatestRelease = releases.find(release => {
return release.tag_name.startsWith(releasePrefix);
});
if (llvmLatestRelease){
let asset = llvmLatestRelease.assets.find(asset =>{
return asset.name.includes(artifactArch);
});
if (!asset){
core.setFailed(`Artifact for '${artifactArch}' not found in release ${llvmLatestRelease.tag_name} (${llvmLatestRelease.html_url})`);
process.exit();
}
return asset.browser_download_url;
}

page++;
} while(releases.length > 0);

core.setFailed(`No LLVM releases with '${releasePrefix}' atifacts found! Please release LLVM before running this workflow.`);
process.exit();

- name: download
shell: bash
run: |
mkdir -p ${{ inputs.dir }}
curl -sSLo llvm.tar.gz ${{ steps.find.outputs.result }}
ls -al

- name: unpack
shell: bash
run: |
tar -xf llvm.tar.gz -C ${{ inputs.dir }} --strip-components=${{ inputs.stripComponents }}
rm llvm.tar.gz
ls -al ${{ inputs.dir }}
55 changes: 18 additions & 37 deletions .github/workflows/build-revive-wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,62 +6,48 @@ on:
branches: ["main"]
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always
REVIVE_WASM_INSTALL_DIR: ${{ github.workspace }}/target/wasm32-unknown-emscripten/release

jobs:
build-revive-wasm:
runs-on: parity-large
runs-on: ubuntu-24.04
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4

- name: Cache LLVM build
id: cache-llvm
uses: actions/cache@v3
with:
path: |
target-llvm/emscripten/target-final
target-llvm/gnu/target-final
# Use a unique key based on LLVM version or configuration files to avoid cache invalidation
key: llvm-build-${{ runner.os }}-${{ hashFiles('LLVM.lock', 'Cargo.toml', 'Cargo.lock', 'crates/llvm-builder/**', '.github/workflows/build-revive-wasm.yml') }}

- name: Install system dependencies
run: |
sudo apt-get update && sudo apt-get install -y cmake ninja-build curl git libssl-dev pkg-config clang lld

- name: Install Rust stable toolchain
uses: actions-rs/toolchain@v1
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
profile: minimal
toolchain: stable
components: rust-src
target: wasm32-unknown-emscripten
rustflags: ""

- name: Install LLVM build dependencies
run: |
make install-llvm-builder
revive-llvm --target-env emscripten clone
- name: get llvm gnu
uses: ./.github/actions/get-llvm
with:
artifactArch: x86_64-linux-gnu
- name: get llvm emscripten
uses: ./.github/actions/get-llvm
with:
artifactArch: emscripten

- name: install emsdk
uses: ./.github/actions/get-emsdk

- name: Setup revive environment variables
run: |
echo "LLVM_SYS_181_PREFIX=$(pwd)/target-llvm/gnu/target-final" >> $GITHUB_ENV
echo "REVIVE_LLVM_TARGET_PREFIX=$(pwd)/target-llvm/emscripten/target-final" >> $GITHUB_ENV

- name: Build host LLVM
if: steps.cache-llvm.outputs.cache-hit != 'true'
run: |
revive-llvm build --llvm-projects lld --llvm-projects clang

- name: Build target LLVM
if: steps.cache-llvm.outputs.cache-hit != 'true'
run: |
source emsdk/emsdk_env.sh
revive-llvm --target-env emscripten build --llvm-projects lld

- run: |
rustup show
cargo --version
Expand All @@ -70,11 +56,6 @@ jobs:
cmake --version
bash --version

- name: Use Cached LLVM
if: steps.cache-llvm.outputs.cache-hit == 'true'
run: |
echo "Using cached LLVM"

- name: Build revive
run: |
source emsdk/emsdk_env.sh
Expand Down
36 changes: 0 additions & 36 deletions .github/workflows/get-release-artifact.js

This file was deleted.

4 changes: 4 additions & 0 deletions .github/workflows/release-llvm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ on:
required: true
description: llvm version in "x.x.x" format, e.g. "18.1.8"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always

Expand Down
Loading
Loading