Skip to content
This repository was archived by the owner on Jan 30, 2023. It is now read-only.

Commit 6ccbaa3

Browse files
committed
Be more agressive about requiring the latest develop
otherwise the builds will just take forever and it is good practice to merge develop frequently to prevent future conflicts.
1 parent 4c023d5 commit 6ccbaa3

File tree

5 files changed

+47
-16
lines changed

5 files changed

+47
-16
lines changed

.ci/build-docker.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ set -ex
2121

2222
# We speed up the build process by copying built artifacts from ARTIFACT_BASE
2323
# during docker build. See /docker/Dockerfile for more details.
24-
ARTIFACT_BASE=${ARTIFACT_BASE:-sagemath/sagemath-dev:latest}
24+
ARTIFACT_BASE=${ARTIFACT_BASE:-sagemath/sagemath-dev:develop}
2525

2626
# Seed our cache with $ARTIFACT_BASE if it exists
2727
docker pull $ARTIFACT_BASE || true

.circleci/config.yml

+13-4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# This file configures automatic builds of Sage on [CircleCI](https://circleci.com).
22
# To make the build time not too excessive, we seed the build cache with
3-
# sagemath/sagemath-dev:latest. When basic SPKGs change, this does not help much
3+
# sagemath/sagemath-dev:develop. When basic SPKGs change, this does not help much
44
# and the full build might exceed CircleCI's limits for open source projcets (five
55
# hours on 2 vCPUs as of early 2018.)
66
# You might want to try to build locally or with GitLab CI, see
@@ -27,15 +27,24 @@ jobs:
2727
# Therefore we roll our own to protect $SECRET_* variables.
2828
. .ci/protect-secrets.sh
2929
30-
export DOCKER_TAG=${CIRCLE_TAG:-$CIRCLE_BRANCH}
3130
# Build docker images
32-
# TODO: Change this line to sagemath/sagemath-dev:latest
33-
export ARTIFACT_BASE=saraedum/sagemath-dev:gitlabci
31+
export DOCKER_TAG=${CIRCLE_TAG:-$CIRCLE_BRANCH}
32+
case $DOCKER_TAG in
33+
"develop" | "latest")
34+
export ARTIFACT_BASE=source-clean
35+
;;
36+
*)
37+
# TODO: Change this line to sagemath/sagemath-dev:develop
38+
export ARTIFACT_BASE=saraedum/sagemath-dev:develop
39+
;;
40+
esac
3441
. .ci/build-docker.sh
42+
3543
# Test that the images work
3644
. .ci/test-dev.sh $DOCKER_IMAGE_DEV
3745
. .ci/test-cli.sh $DOCKER_IMAGE_CLI
3846
. .ci/test-jupyter.sh $DOCKER_IMAGE_CLI localhost
47+
3948
# Push docker images to dockerhub if a dockerhub user has been configured
4049
. .ci/push-dockerhub.sh sagemath-dev
4150
. .ci/push-dockerhub.sh sagemath

.gitlab-ci.yml

+6-3
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# This file configures automatic builds of Sage on [GitLab](https://gitlab.com).
22
# To make the build time not too excessive, we seed the build cache with
3-
# sagemath/sagemath-dev:latest. When basic SPKGs changed, this does not help
3+
# sagemath/sagemath-dev:develop. When basic SPKGs changed, this does not help
44
# much and the full build might the set time limit in GitLab. You can increase
55
# that limit in Settings → CI/CD.
66
# You can also provision your own private more powerful runner in the same
@@ -18,8 +18,8 @@ variables:
1818
DOCKER_TAG: $CI_COMMIT_REF_SLUG
1919
# Builds are very I/O intensive; make sure we have a fast file system.
2020
DOCKER_DRIVER: overlay2
21-
# TODO: Change this line to sagemath/sagemath-dev:latest
22-
ARTIFACT_BASE: saraedum/sagemath-dev:gitlabci
21+
# TODO: Change this line to sagemath/sagemath-dev:develop
22+
ARTIFACT_BASE: saraedum/sagemath-dev:develop
2323

2424
before_script:
2525
# GitLab has no mechanism yet to hide secret variables: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784
@@ -50,6 +50,7 @@ build-from-latest: &build
5050
- . .ci/push-gitlab.sh sagemath
5151
except:
5252
- master
53+
- develop
5354

5455
# Build Sage and its documentation from a clean checkout of Sage.
5556
# Note that this takes a very long time. You probably want to run this on your
@@ -60,6 +61,8 @@ build-from-clean:
6061
ARTIFACT_BASE: "source-clean"
6162
only:
6263
- master
64+
- develop
65+
except: []
6366

6467
test-dev:
6568
stage: test

docker/Dockerfile

+26-7
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,12 @@
1313
# docker build without any external orchestration script. #
1414
# #
1515
# The idea to achieve (1) is to reuse the build artifacts from the latest #
16-
# master build. This is slightly against the philosophy of a Dockerfile (which #
16+
# develop build. This is slightly against the philosophy of a Dockerfile (which#
1717
# should produce perfectly reproducible outputs) but building Sage from scratch#
1818
# just takes too long at the moment to do this all the time. ARTIFACT_BASE #
1919
# controls which build artifacts are used. You probably want to set this to #
20-
# sagemath/sagemath-dev:latest which takes the latest build from the official #
21-
# master branch. The default is source-clean which builds Sage from scratch. #
20+
# sagemath/sagemath-dev:develop which takes the latest build from the official #
21+
# develop branch. The default is source-clean which builds Sage from scratch. #
2222
# If you want to understand how this works, have a look at source-from-context #
2323
# which merges ARTIFACT_BASE with the context, i.e., the contents of the sage #
2424
# source directory. #
@@ -92,15 +92,34 @@ RUN git remote add trac [email protected]:sage.git
9292
FROM $ARTIFACT_BASE as source-from-context
9393
WORKDIR $HOME
9494
COPY --chown=sage:sage . sage-context
95-
# Checkout the commit that is in $HOME/sage-context
95+
# Checkout the commit that checked out in $HOME/sage-context
96+
# This is a bit complicated because our local .git/ is empty and we want to
97+
# make sure that we only change the mtimes of a minimal number of files.
98+
# 1) Restore the git checkout ARTIFACT_BASE was built from, recorded in
99+
# docker/commit. (Or go directly to FETCH_HEAD if there is no history to
100+
# restore.)
101+
# 2) Merge in FETCH_HEAD but only if it is a fast-forward, i.e., if it is an
102+
# ancestor of the commit restored in 1. If we would not do that we would get
103+
# a new commit hash in docker/commit that is not known outside of this build
104+
# run. Since docker/commit was in the history of FETCH_HEAD this should
105+
# automatically be a fast-forward.
106+
# 3) Trash .git again to save some space.
96107
ARG SAGE_ROOT=/home/sage/sage
97108
WORKDIR $SAGE_ROOT
98109
RUN git fetch "$HOME/sage-context" HEAD \
99-
&& git reset FETCH_HEAD \
100-
&& git checkout -f FETCH_HEAD \
110+
&& if [ -e docker/commit ]; then \
111+
git reset `cat docker/commit` \
112+
|| ( echo "Could not find commit `cat docker/commit` in your local Git history. Please merge in the latest develop branch to fix this: git fetch trac && git merge trac/develop." && exit 1 ) \
113+
else \
114+
echo "You are building from $ARTIFACT_BASE which has no docker/commit file. That's a bug unless you are building from source-clean or something similar." \
115+
&& git reset FETCH_HEAD \
116+
&& git checkout -f FETCH_HEAD; \
117+
fi \
118+
&& git merge --ff-only FETCH_HEAD \
101119
&& git log -1 --format=%H > docker/commit \
102120
&& rm -rf .git
103-
# Copy over all the untracked/staged/unstaged changes from sage-context
121+
# Copy over all the untracked/staged/unstaged changes from sage-context. This
122+
# is relevant for non-CI invocations of this Dockerfile.
104123
WORKDIR $HOME/sage-context
105124
RUN if git status --porcelain | read CHANGES; then \
106125
git -c user.name=docker-build -c [email protected] stash -u \

docker/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ There are several flavours of this image.
2626
```
2727
* [`sagemath/sagemath-dev`![image size](https://img.shields.io/microbadger/image-size/sagemath/sagemath-dev.svg)](https://hub.docker.com/sagemath/sagemath-dev) contains all the build artifacts to rebuild Sage quickly. This version is probably only relevant for Sage developers. Run this image with:
2828
```
29-
docker run -it sagemath/sagemath-dev:latest
29+
docker run -it sagemath/sagemath-dev:develop
3030
```
3131
This triggers a rebuild and drops you in a shell afterwards. Note that the git repository has been emptied to save space. If you want to use git, fetch from your git repository with `git fetch trac` and go to the commit that was used to create this image with
3232
```

0 commit comments

Comments
 (0)