Skip to content

Commit

Permalink
Upgrade dev-cmd and simplify slightly. (#126)
Browse files Browse the repository at this point in the history
Although uv broke backwards compatibility in its lock format for
editable projects necessitating otherwise unrelated workflow 
gymnastics.

C.F.: astral-sh/uv#10622 (comment)
  • Loading branch information
jsirois authored Jan 17, 2025
1 parent 0e9e616 commit 2bd9199
Show file tree
Hide file tree
Showing 5 changed files with 77 additions and 26 deletions.
44 changes: 34 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,22 +63,36 @@ jobs:
steps:
- name: Install the latest version of uv
if: matrix.docker-platform == '' && matrix.os != 'windows-arm64'
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@v5
- name: Setup uv
if: matrix.docker-platform == '' && matrix.os != 'windows-arm64'
run: |
export UV="$(which uv)"
"${UV}" -V
echo UV="${UV}" >> ${GITHUB_ENV}
- name: Install the latest version of uv
if: matrix.os == 'windows-arm64'
shell: cmd
run: |
"C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\VsDevCmd.bat"
cargo install --git https://github.com/astral-sh/uv uv
uv -V
CI_BIN_DIR="$(mktemp -d)"
echo CI_BIN_DIR="${CI_BIN_DIR}" >> ${GITHUB_ENV}
(
cd "${CI_BIN_DIR}"
curl -fL -O https://github.com/astral-sh/uv/releases/latest/download/uv-x86_64-pc-windows-msvc.zip
curl -fL -O https://github.com/astral-sh/uv/releases/latest/download/uv-x86_64-pc-windows-msvc.zip.sha256
shasum -c uv-x86_64-pc-windows-msvc.zip.sha256
unzip uv-x86_64-pc-windows-msvc.zip
)
export UV="${CI_BIN_DIR}/uv.exe"
"${UV}" -V
echo UV="${UV}" >> ${GITHUB_ENV}
- name: Setup x86_64 Python for Prism
if: matrix.os == 'windows-arm64'
run: |
# N.B.: We use an x86-64 Python for Windows ARM64 because this is what we ship with via
# PBS, and we need to be able to resolve x86-64 compatible requirements (which include
# native deps like psutil) for our shiv.
UV_PYTHON_VERSION=cpython-3.12.8-windows-x86_64-none
uv python install ${UV_PYTHON_VERSION}
"${UV}" python install ${UV_PYTHON_VERSION}
echo UV_PYTHON="${UV_PYTHON_VERSION}" >> ${GITHUB_ENV}
- name: Installing emulators
if: matrix.docker-platform != ''
Expand All @@ -87,7 +101,8 @@ jobs:
uses: actions/checkout@v4
- name: Check Formatting & Lints
if: matrix.docker-platform == ''
run: uv run dev-cmd ci --skip test
run: |
"${UV}" run dev-cmd ci --skip test
- name: Check Formatting & Lints
if: matrix.docker-platform != ''
run: |
Expand All @@ -112,7 +127,8 @@ jobs:
echo SCIE_BASE=C:/tmp/gha/nce >> ${GITHUB_ENV}
- name: Unit Tests
if: matrix.docker-platform == ''
run: uv run dev-cmd test -- -vvs
run: |
"${UV}" run dev-cmd test -- -vvs
- name: Unit Tests
if: matrix.docker-platform != ''
run: |
Expand Down Expand Up @@ -142,7 +158,8 @@ jobs:
python:3.12-bookworm bash _ci_test.sh
- name: Build & Package
if: matrix.docker-platform == ''
run: uv run dev-cmd package
run: |
"${UV}" run dev-cmd package
- name: Build & Package
if: matrix.docker-platform != ''
run: |
Expand All @@ -161,7 +178,8 @@ jobs:
"
- name: Generate Doc Site
if: matrix.docker-platform == ''
run: uv run dev-cmd doc linkcheck
run: |
"${UV}" run dev-cmd doc linkcheck
- name: Generate Doc Site
if: matrix.docker-platform != ''
run: |
Expand All @@ -178,3 +196,9 @@ jobs:
adduser --disabled-password --gecos '' --gid $(id -g) --uid $(id -u) build &&
su build -c 'uv run dev-cmd doc linkcheck'
"
- name: Cleanup
if: always()
run: |
if [[ -n "${CI_BIN_DIR}" && -d "${CI_BIN_DIR}" ]]; then
rm -rf "${CI_BIN_DIR}"
fi
35 changes: 28 additions & 7 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,22 +86,36 @@ jobs:
steps:
- name: Install the latest version of uv
if: matrix.docker-platform == '' && matrix.os != 'windows-arm64'
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@v5
- name: Setup uv
if: matrix.docker-platform == '' && matrix.os != 'windows-arm64'
run: |
export UV="$(which uv)"
"${UV}" -V
echo UV="${UV}" >> ${GITHUB_ENV}
- name: Install the latest version of uv
if: matrix.os == 'windows-arm64'
shell: cmd
run: |
"C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\VsDevCmd.bat"
cargo install --git https://github.com/astral-sh/uv uv
uv -V
CI_BIN_DIR="$(mktemp -d)"
echo CI_BIN_DIR="${CI_BIN_DIR}" >> ${GITHUB_ENV}
(
cd "${CI_BIN_DIR}"
curl -fL -O https://github.com/astral-sh/uv/releases/latest/download/uv-x86_64-pc-windows-msvc.zip
curl -fL -O https://github.com/astral-sh/uv/releases/latest/download/uv-x86_64-pc-windows-msvc.zip.sha256
shasum -c uv-x86_64-pc-windows-msvc.zip.sha256
unzip uv-x86_64-pc-windows-msvc.zip
)
export UV="${CI_BIN_DIR}/uv.exe"
"${UV}" -V
echo UV="${UV}" >> ${GITHUB_ENV}
- name: Setup x86_64 Python for Prism
if: matrix.os == 'windows-arm64'
run: |
# N.B.: We use an x86-64 Python for Windows ARM64 because this is what we ship with via
# PBS, and we need to be able to resolve x86-64 compatible requirements (which include
# native deps like psutil) for our shiv.
UV_PYTHON_VERSION=cpython-3.12.8-windows-x86_64-none
uv python install ${UV_PYTHON_VERSION}
"${UV}" python install ${UV_PYTHON_VERSION}
echo UV_PYTHON_ARGS="--python ${UV_PYTHON_VERSION}" >> ${GITHUB_ENV}
- name: Installing emulators
if: matrix.docker-platform != ''
Expand All @@ -112,7 +126,8 @@ jobs:
ref: ${{ needs.determine-tag.outputs.release-tag }}
- name: Package science ${{ needs.determine-tag.outputs.release-tag }} binary
if: matrix.docker-platform == ''
run: uv run ${UV_PYTHON_ARGS} dev-cmd package
run: |
"${UV}" run ${UV_PYTHON_ARGS} dev-cmd package
- name: Package science ${{ needs.determine-tag.outputs.release-tag }} binary
if: matrix.docker-platform != ''
run: |
Expand Down Expand Up @@ -151,3 +166,9 @@ jobs:
files: dist/science-*
fail_on_unmatched_files: true
discussion_category_name: Announcements
- name: Cleanup
if: always()
run: |
if [[ -n "${CI_BIN_DIR}" && -d "${CI_BIN_DIR}" ]]; then
rm -rf "${CI_BIN_DIR}"
fi
7 changes: 5 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,9 @@ line-length = 100
[tool.ruff.lint]
extend-select = ["I"]

[tool.uv]
required-version = ">=0.5.19"

[tool.dev-cmd.commands]
check-python-version = ["scripts/check-python-version-alignment.py"]

Expand All @@ -98,8 +101,7 @@ check-lint = ["ruff", "check"]

type-check = ["mypy", "docs/_ext", "science", "scripts", "setup.py", "tests", "test-support"]

doc = ["sphinx-build", "-b", "html", "-aEW", "docs", "docs/build/html"]
linkcheck = ["sphinx-build", "-b", "linkcheck", "-aEW", "docs", "docs/build/linkcheck"]
doc = ["sphinx-build", "-b", "{-type:html}", "-aEW", "docs", "docs/build/{-type:html}"]

create-zipapp = ["scripts/create-zipapp.py"]

Expand Down Expand Up @@ -156,6 +158,7 @@ PYTHONPATH = "../test-support"
SCIENCE_TEST_PYZ_PATH = "../dist/science.pyz"

[tool.dev-cmd.tasks]
linkcheck = ["doc-type:linkcheck"]
test = ["create-zipapp", "pytest"]
checks = [["check-python-version", ["fmt", "lint"]], "type-check", "test"]
ci = [["check-python-version", "check-fmt", "check-lint", "type-check"], "test"]
Expand Down
10 changes: 7 additions & 3 deletions scripts/create-zipapp.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
# Licensed under the Apache License, Version 2.0 (see LICENSE).

import atexit
import os
import shutil
import subprocess
import sys
Expand All @@ -13,13 +14,16 @@ def main() -> int:
work_dir = Path(tempfile.mkdtemp(prefix="science-zipapp-build."))
atexit.register(shutil.rmtree, work_dir, ignore_errors=True)

# An affordance for CI uv setups on some shards.
uv = os.environ.get("UV", "uv")

locked_requirements = work_dir / "requirements.txt"
wheels = work_dir / "wheels"
processes = [
subprocess.Popen(
args=["uv", "-q", "export", "--no-dev", "--no-emit-project", "-o", locked_requirements],
args=[uv, "-q", "export", "--no-dev", "--no-emit-project", "-o", locked_requirements],
),
subprocess.Popen(args=["uv", "-q", "build", "--wheel", "-o", wheels]),
subprocess.Popen(args=[uv, "-q", "build", "--wheel", "-o", wheels]),
]
while processes:
process = processes.pop()
Expand All @@ -33,7 +37,7 @@ def main() -> int:
if 0 != (
exit_code := subprocess.call(
args=[
"uv",
uv,
"-q",
"pip",
"install",
Expand Down
7 changes: 3 additions & 4 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 2bd9199

Please sign in to comment.