Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apptainer CI #53

Merged
merged 25 commits into from
Apr 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
b77c471
Changed job name for deployment on PyPI.
jlenain Apr 7, 2023
b820514
Add a CI for building Apptainer container image and pushing it to the…
jlenain Apr 7, 2023
8351d2e
Try fixing the login mechanism to GitHub Container Registry.
jlenain Apr 7, 2023
93e9611
Try again fixing the login mechanism to GitHub Container Registry.
jlenain Apr 7, 2023
d6897ce
Fixed typo when logging into GitHub Container Registry, added logout …
jlenain Apr 7, 2023
3f95e39
Try pushing container image to my own repo.
jlenain Apr 7, 2023
bd39f33
Assign specific job permission for usage of GITHUB_TOKEN, see https:/…
jlenain Apr 7, 2023
d49db20
Fix workflow permissions
jlenain Apr 7, 2023
bfc83a1
Fix workflow permissions
jlenain Apr 7, 2023
2d0014f
Update deploy-ghcr.yml
jlenain Apr 7, 2023
c5e1353
Try to fix deploy-container workflow.
jlenain Apr 7, 2023
3341815
Update deploy-ghcr.yml
jlenain Apr 7, 2023
fe6a585
Update deploy-ghcr.yml
jlenain Apr 7, 2023
9aed313
Update deploy-ghcr.yml
jlenain Apr 7, 2023
88b2557
Update deploy-ghcr.yml
jlenain Apr 7, 2023
259aa4d
Try to fix deploy-container workflow.
jlenain Apr 7, 2023
afb55bd
Fix login to GHCR
jlenain Apr 7, 2023
6142560
Fix typo in deploy
jlenain Apr 7, 2023
07e3813
Changed login/logout mechanism to GHCR using apptainer remote.
jlenain Apr 7, 2023
25622f3
Simplify Apptainer definition file by taking the local environment.ym…
jlenain Apr 7, 2023
dbc08ec
Change triggering events to production mode, don't deploy container o…
jlenain Apr 7, 2023
333b92b
Still need to git clone nectarchain within container at build.
jlenain Apr 7, 2023
5c960ce
Use local repo files gathered from checkout action, instead of git cl…
jlenain Apr 8, 2023
3f82ae8
But what is the relative path of environment.yml ?
jlenain Apr 8, 2023
9a2af1f
Properly copy local repo files gathered by checkout action into the c…
jlenain Apr 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/deploy-ghcr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Build and deploy as Apptainer container image to GitHub Container Registry

on:
pull_request: []
push:
branches:
- master
release:
types: [ published ]

jobs:
build-deploy-container:
runs-on: ubuntu-latest

defaults:
run:
shell: bash -leo pipefail {0}

permissions:
packages:
write

strategy:
matrix:
deffiles: [[singularity/Singularity, latest]]

env:
container: nectarchain
registry: ghcr.io

steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 1

- name: Setup Apptainer
uses: eWaterCycle/setup-apptainer@v2
with:
apptainer-version: 1.1.7

- name: Build ${{ matrix.deffiles[1] }}
run: |
echo "Preparing to build ${{ env.container }} from ${{ matrix.deffiles[0] }}"
if [ ! -f "${{ matrix.deffiles[0]}}" ]; then
echo "Singularity definition file ${{ matrix.deffiles[0] }} does not exist"
exit 1
fi
apptainer build ${{ env.container }}.sif ${{ matrix.deffiles[0] }}
ls -lh

- name: Login to GitHub Container Registry
# Don't log into registry on pull request.
if: github.event_name != 'pull_request'
run: |
echo ${{ github.token }} | apptainer remote login --username ${{ github.actor }} --password-stdin oras://${{ env.registry }}

- name: Deploy ${{ matrix.deffiles[1] }}
# Don't push the container on a pull request.
if: github.event_name != 'pull_request'
run: |
apptainer push ${{ env.container }}.sif oras://${{ env.registry }}/${{ github.repository }}:${{ matrix.deffiles[1] }}

- name: Post Login to GitHub Container Registry
# Don't log out from registry on pull request.
if: github.event_name != 'pull_request'
run: |
apptainer remote logout oras://${{ env.registry }}
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
- 'v*'

jobs:
deploy:
deploy-pypi:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand Down
30 changes: 15 additions & 15 deletions singularity/Singularity
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,23 @@
# Built from mambaforge, with special conda environment containing nectarchain
#
# Jean-Philippe Lenain <[email protected]>
# Time-stamp: "2023-04-06 14:42:19 jlenain"
# Time-stamp: "2023-04-08 00:20:24 jlenain"
#
# Typically, build this image with:
# `sudo singularity build nectarchain.sif Singularity`
# `sudo apptainer build nectarchain.sif singularity/Singularity`
#
# Then, typically run an instance of this image with:
# `singularity shell nectarchain.sif`
# `apptainer shell nectarchain.sif`

Bootstrap: docker
From: condaforge/mambaforge

%setup
mkdir -p ${SINGULARITY_ROOTFS}/opt/cta/nectarchain

%files
. /opt/cta/nectarchain

# From https://github.com/hpcng/singularity/issues/5075#issuecomment-594391772
%environment
action="${0##*/}"
Expand All @@ -27,9 +33,7 @@ From: condaforge/mambaforge
fi

%post
ORIG=$PWD

# CA certificates
# Install CA certificates
apt -y update
# cf. https://serverfault.com/a/992421
DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt -y install software-properties-common curl
Expand All @@ -40,24 +44,20 @@ From: condaforge/mambaforge
. /opt/conda/etc/profile.d/conda.sh
. /opt/conda/etc/profile.d/mamba.sh
mamba update --quiet --name base conda mamba
mkdir -p /opt/cta
cd /opt/cta

# Install nectarchain
git clone https://github.com/cta-observatory/nectarchain.git
mamba env create --quiet --file nectarchain/environment.yml --prefix /opt/conda/envs/nectarchain
mamba env create --quiet --file /opt/cta/nectarchain/environment.yml --prefix /opt/conda/envs/nectarchain
mamba activate nectarchain
cd nectarchain
cd /opt/cta/nectarchain
pip install -e .

# Optionally install and configure DIRAC:
mamba install -y -c conda-forge dirac-grid
mamba install --quiet -y -c conda-forge dirac-grid
conda env config vars set X509_CERT_DIR=${CONDA_PREFIX}/etc/grid-security/certificates X509_VOMS_DIR=${CONDA_PREFIX}/etc/grid-security/vomsdir X509_VOMSES=${CONDA_PREFIX}/etc/grid-security/vomses
mamba deactivate
mamba activate nectarchain
pip install CTADIRAC
pip install COMDIRAC

pip install CTADIRAC COMDIRAC

# Since there is no proxy available at build time, manually configure the CTADIRAC client
cat <<EOF > ${CONDA_PREFIX}/etc/dirac.cfg
DIRAC
Expand Down