Request adoptopenjdk & Temurin container build images be published to alternate hosting site

[AdamBrousseau]

At the moment, the build images [1][2] are published to Docker Hub.
The OpenJ9 Community farm and IBM Semeru Runtimes farm pull these images for our Linux build environments for x64 (amd64), ppc64le and aarch64 (arm64). Docker Hub has recently implemented a pricing change [3] that negatively impacts these pulls. We are requesting Adoptium to consider publishing these to an alternate source. Here are a few alternatives I have found.

• Amazon
• GitHub
• IBM
• Quay
• Red Hat

[1] https://hub.docker.com/r/adoptopenjdk/centos7_build_image
[2] https://hub.docker.com/r/adoptopenjdk/centos6_build_image
[3] https://www.docker.com/blog/november-2024-updated-plans-announcement/

[sxa]

Related:

• Identify suitable off-machine location for storing private build container images #3217
• add test docker image to ghcr for dynamic VM testing #3775 (ghcr.io)
• docker: add action to deploy windows build image #3780 (adoptium.azurecr.io)

[tellison]

This was discussed at the PMC meeting on Dec 4. The plan is to proceed with designing alternate location/locations suitable for build and delivery images.

[sxa]

We shall look at publishing our images to ghcr.io and then mirroring them elsewhere (e.g. azure CR for performance) as desired.

[sxa]

@gdams Following on from the PMC discussion we had earlier this month, what do we need to do to set up a suitable registry associated with the temurin-build repo that can be used for public and private images?

[smlambert]

Updating the title of this one to include the Temurin images also and moving this to the iteration directly after the Jan release.

[sxa]

@smlambert Moving of the Temurin images should be a separate task and not covered under this issue IMHO as the solution will not necessarily be the same, particularly with respect to permissions of certain images.

[sxa]

Proposal for build images to replace adoptopenjdk/*_build_image

• Create a GitHub packages registry in the Adoptium org
• Use a build_image image which can be used for public build images, which can have tags appropriately e.g. ghcr.io/adoptium/build_image:centos7
• Use a build_image_private (or build_image_restricted) which can be used for non-public images such as those for windows and Linux/s390x.
• Both of these can have their access set independently (public vs private) and both should either be associated with the temurin-build or infrastructure repositories.
• Any Windows images should also be published up to azurecr as per docker: add action to deploy windows build image #3780 for performance reasons although we should endeavour to treat the gchr one as the master location for the images for consistency, and then others can mirror it to any of the alternate repositories listed in the initial description for convenience if desired.

This will require changes to https://ci.adoptium.net/job/centos7_docker_image_updater/ - it's probably about time we renamed that job anyway so this will provide a good opportunity to set up another job in parallel ;-)

[sxa]

Initial image for prototyping is at ghcr.io/sxa/build_image:centos7_linux-arm64 (Linux/aarch64 - others are being trialled but that's the one I've been experimenting with mostly). Obviously that won't be the final location as it's tied to my account but it is suitable for verifying that the image from somewhere else works ;-)

Sample build from this location: https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-aarch64-temurin/482

[sxa]

Job https://ci.adoptium.net/job/build_image_updater/4 has successfully published the "normal" set of build images excluding centos6 (Due to #3839) to https://github.com/users/sxa/packages/container/package/build_image with a suitable multiarch manifest so you can now pull the image listed above without the platform suffix e.g. ghcr.io/sxa/build_image:centos7

[sxa]

I have an updated token for Eclipse which allows me to push to ghcr.io/adoptium/adoptium_build_image - will continue testing before

Also related: #2734 which I will try and implement at the same time.

[sxa]

Published images look ok. The packages are currently private but I've spoken to @pstankie relating to what is needed to make them public. I'm trialling a jdk17u/linux/x64 build pipeline with the new containers which are being pulled with our credential (WITHHELD is replaced with the name of the credential as stored in our jenkins instance):

  "DOCKER_IMAGE": "ghcr.io/adoptium/adoptium_build_image:centos7_linux-amd64",
  "DOCKER_ARGS": "",
  "DOCKER_NODE": "",
  "DOCKER_REGISTRY": "https://ghcr.io",
  "DOCKER_CREDENTIAL": "WITHHELD",

This mechanism should be suitable on a more permanent basis for retrieving container images which have to be restricted or licensing reasons such as RHEL for s390x or Windows.

[sxa]

Raised https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/5724 to get the images adjusted so they do not require authentication to pull them down (Noting that the pipeline referenced in the previous comment did use authentication and managed to pull the image although it seemed to try and issue a docker tag operations afterwards which failed)

[sxa]

@AdamBrousseau Images are now public at https://github.com/orgs/adoptium/packages/container/package/adoptium_build_image so can be pulled using the parameters in the earlier comment excluding the DOCKER_CREDENTIAL 👍🏻

[AdamBrousseau]

Thanks, @sxa . I will give it a try and report back if we have any problems.