Use multi stage build to clear artifacts and reduce image size #90

[mickare]

Suggestion by @pranas: #90 (comment)

It reduces the image size from 112MB to 86MB.

[tianon]

docker-library/official-images#3383 😞

[jouve]

Hi,

this will not work for binaries with setuid bit because COPY does not keep it:

docker build --no-cache . -t jouve/test
Sending build context to Docker daemon  39.31MB
Step 1/6 : FROM scratch as base
 ---> 
Step 2/6 : ADD ubuntu-artful-core-cloudimg-amd64-root.tar.gz /
 ---> 3cbb692107ee
Step 3/6 : RUN ls -l /usr/bin/passwd
 ---> Running in c309e026a953
-rwsr-xr-x 1 root root 54224 Aug 20  2017 /usr/bin/passwd
Removing intermediate container c309e026a953
 ---> 6975740163a1
Step 4/6 : FROM scratch
 ---> 
Step 5/6 : COPY --from=base / /
 ---> fb9f22eb3121
Step 6/6 : RUN ls -l /usr/bin/passwd
 ---> Running in 9ce611e2a914
-rwxr-xr-x 1 root root 54224 Aug 20  2017 /usr/bin/passwd
Removing intermediate container 9ce611e2a914
 ---> 60931d907a8e
Successfully built 60931d907a8e
Successfully tagged jouve/test:latest

[tianon]

Indeed, it also strips all UID/GID bits. 😞

[mickare]

😞

Thanks @jouve, that is a very good point and a serious problem with my proposal. 👍
I hope it is okay that I cited you here: docker-library/official-images#3383 (comment)

[tianon]

Arg, I wanted to reconsider this approach thanks to moby/moby#38599 (which is available in 19.03+!), but the "max 5 builds" policy on https://partner-images.canonical.com/core/ makes it untenable.

For example, in https://partner-images.canonical.com/core/groovy/ we barely have more than a week's worth of builds, which means that in between our ~monthly Ubuntu updates, ubuntu:groovy would officially fail to build for more than half the month... 🤦 😞

Edit: I guess that wouldn't actually matter for the proposal in this PR, but the main reason I'm interested in multi-stage builds here is specifically because we could use it to stop committing tarballs to the repository (not really because it would allow us to squash -- the layers in this image don't cause significant downstream harm now that /var/lib/apt/lists is empty in the release tarballs).

[tianon]

For reference, here's what I've been playing with -- I started to port all the current tweaks to separate lines here, and realized that if I instead just pulled in debuerreotype itself and ran the appropriate script, the generated files would also include the long-form configuration explanations I spent so much time writing (https://github.com/debuerreotype/debuerreotype/blob/0.10/scripts/debuerreotype-minimizing-config):

FROM debian:buster-slim AS fetch

RUN set -ex; \
	apt-get update; \
	apt-get install -y --no-install-recommends \
		ca-certificates \
		gnupg dirmngr \
		wget \
	; \
	rm -rf /var/lib/apt/lists/*

# install debuerreotype (https://github.com/debuerreotype/debuerreotype) for making Docker-specific tweaks in a consistent way
ENV DEBUERREOTYPE_VERSION 0.10
RUN set -eux; \
	wget -O debuerreotype.tgz "https://github.com/debuerreotype/debuerreotype/archive/$DEBUERREOTYPE_VERSION.tar.gz"; \
	mkdir /opt/debuerreotype; \
	tar -xvf debuerreotype.tgz -C /opt/debuerreotype --strip-components=1; \
	rm debuerreotype.tgz; \
# "debuerreotype-chroot" (unreasonably) unconditionally assumes it can do "unshare" and "mount --rbind" so we need to replace it with just "chroot"
	ln -svfT "$(command -v chroot)" /opt/debuerreotype/scripts/debuerreotype-chroot
ENV PATH /opt/debuerreotype/scripts:$PATH

ENV UBUNTU_FINGERPRINT D2EB44626FDDC30B513D5BB71A5D6C4C7DB87C81

RUN gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$UBUNTU_FINGERPRINT"

ENV UBUNTU_SUITE focal
ENV UBUNTU_SERIAL 20200703

RUN set -eux; \
	\
	wget "https://partner-images.canonical.com/core/${UBUNTU_SUITE}/${UBUNTU_SERIAL}/SHA256SUMS"; \
	wget "https://partner-images.canonical.com/core/${UBUNTU_SUITE}/${UBUNTU_SERIAL}/SHA256SUMS.gpg"; \
	gpg --batch --verify SHA256SUMS.gpg SHA256SUMS; \
	\
	arch="$(dpkg --print-architecture)"; \
	tarball="ubuntu-${UBUNTU_SUITE}-core-cloudimg-${arch}-root.tar.gz"; \
	wget -O "$tarball" "https://partner-images.canonical.com/core/${UBUNTU_SUITE}/${UBUNTU_SERIAL}/$tarball" --progress=dot:giga; \
	grep -q " *$tarball\$" SHA256SUMS; \
	grep " *$tarball\$" SHA256SUMS | sha256sum -c -; \
	\
	mkdir /rootfs; \
	tar -xf "$tarball" -C /rootfs; \
	\
# verify that the APT lists files do not exist
	[ -z "$(chroot /rootfs apt-get indextargets)" ]; \
# (see https://bugs.launchpad.net/cloud-images/+bug/1699913)
	\
# a few minor docker-specific tweaks
# https://github.com/debuerreotype/debuerreotype/blob/0.10/scripts/debuerreotype-minimizing-config
	debuerreotype-minimizing-config /rootfs; \
	\
# make systemd-detect-virt return "docker"
# See: https://github.com/systemd/systemd/blob/aa0c34279ee40bce2f9681b496922dedbadfca19/src/basic/virt.c#L434
	mkdir -p /rootfs/run/systemd; \
	echo 'docker' > /rootfs/run/systemd/container; \
	\
# fix up any errant timestamps for build reproducibility
# https://github.com/debuerreotype/debuerreotype/blob/0.10/scripts/debuerreotype-fixup#L36-L38
	find /rootfs -newer "$tarball" -exec touch --no-dereference --reference="$tarball" '{}' +

FROM scratch
COPY --from=fetch /rootfs /
CMD ["bash"]