fix: handle docker's unknown/unknown platform in index manifests #975
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jonjohnsonjr
approved these changes
Mar 8, 2023
There was a problem hiding this comment.
This LGTM, but we should consider looking at what the go tool thinks are valid GOOS and GOARCH combinations per the spec.
|
this is so cool! |
4 tasks
listx
pushed a commit
to listx/test-infra
that referenced
this pull request
Aug 18, 2023
The motivation for this change comes from the strange "unsupported
GOOS/GOARCH pair unknown/unknown" errors we were getting in [1] for the
pull-test-infra-prow-image-build-test job. There, we just bumped the
base `git` image from
gcr.io/k8s-prow/git:v20220215-ddc3ad9
to
gcr.io/k8s-prow/git:v20230817-0485b825c2
but this resulted in the aforementioned error for the pod utility
images (clonerefs, sidecar, etc). Digging further, these images are set
to the "all" platform in /.ko.yaml, and this results in the
hack/prowimagebuilder passing the "--platform=all" flag to ko, which
does the actual Docker image build. This instructs ko to look at the
Docker manifest of the base image (in this case the
`gcr.io/k8s-prow/git:...` image), and look through all of the
architectures there, and passes these as is to golang for the go build
invocation [2], which states
To build and push an image for all platforms supported by the
configured base image, simply add --platform=all. This will instruct
ko to look up all the supported platforms in the base image, execute
GOOS=<os> GOARCH=<arch> GOARM=<variant> go build for each platform,
and produce a manifest list containing an image for each platform.
Previously (at least up to gcr.io/k8s-prow/git:v20230111-cd1b3caf9c),
the above behavior worked fine, because the <os> and <arch> bits in the
Docker manifest for these images looked normal (e.g., "linux", "amd64").
For example, gcr.io/k8s-prow/git:v20220215-ddc3ad9 only has entries that
look like
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:232320cd437e5171fa7e29738e9efa191f714da1ae47d96c1f3b7e3016d15e52",
"size": 1363,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
and also for other architectures like "arm64".
But since gcr.io/k8s-prow/git:v20230324-76cde35b3d the git base image
started having "attestation-manifest" annotations [3] in the Docker
manifest that look like
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:5fd1fcaec895346cef283fdf47cf3b061ce7fb4cc8554151eb7b7052ab9886b6",
"size": 566,
"annotations": {
"vnd.docker.reference.digest": "sha256:707be29a6b0a747163add36cb5a594580b4b9dcc7d55dd9324128f8a83d74eb3",
"vnd.docker.reference.type": "attestation-manifest"
},
"platform": {
"architecture": "unknown",
"os": "unknown"
}
},
This means that ko, when given `--platform=all`, will try to pass in a
GOOS "unknown" and GOARCH "unknown" pair to `go build`, which leads to
the image build errors we saw in [1]. This buggy behavior of ko was
fixed in [4], and this change to update ko pulls in that fix. We ran
go get -u github.com/google/ko
go mod tidy
to create this change.
[1] kubernetes#30410
[2] https://ko.build/features/multi-platform/
[3] https://docs.docker.com/build/attestations/attestation-storage/
[4] ko-build/ko#975
listx
pushed a commit
to listx/test-infra
that referenced
this pull request
Aug 19, 2023
The motivation for this change comes from the strange "unsupported
GOOS/GOARCH pair unknown/unknown" errors we were getting in [1] for the
pull-test-infra-prow-image-build-test job. There, we just bumped the
base `git` image from
gcr.io/k8s-prow/git:v20220215-ddc3ad9
to
gcr.io/k8s-prow/git:v20230817-0485b825c2
but this resulted in the aforementioned error for the pod utility
images (clonerefs, sidecar, etc). Digging further, these images are set
to the "all" platform in /.ko.yaml, and this results in the
hack/prowimagebuilder passing the "--platform=all" flag to ko, which
does the actual Docker image build. This instructs ko to look at the
Docker manifest of the base image (in this case the
`gcr.io/k8s-prow/git:...` image), and look through all of the
architectures there, and passes these as is to golang for the go build
invocation [2], which states
To build and push an image for all platforms supported by the
configured base image, simply add --platform=all. This will instruct
ko to look up all the supported platforms in the base image, execute
GOOS=<os> GOARCH=<arch> GOARM=<variant> go build for each platform,
and produce a manifest list containing an image for each platform.
Previously (at least up to gcr.io/k8s-prow/git:v20230111-cd1b3caf9c),
the above behavior worked fine, because the `<os>` and `<arch>` bits in the
Docker manifest for these images looked normal (e.g., "linux", "amd64").
For example, gcr.io/k8s-prow/git:v20220215-ddc3ad9 only has entries that
look like
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:232320cd437e5171fa7e29738e9efa191f714da1ae47d96c1f3b7e3016d15e52",
"size": 1363,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
and also for other architectures like "arm64".
But since gcr.io/k8s-prow/git:v20230324-76cde35b3d the git base image
started having additional "attestation-manifest" annotations [3] in the
Docker manifest that look like
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:5fd1fcaec895346cef283fdf47cf3b061ce7fb4cc8554151eb7b7052ab9886b6",
"size": 566,
"annotations": {
"vnd.docker.reference.digest": "sha256:707be29a6b0a747163add36cb5a594580b4b9dcc7d55dd9324128f8a83d74eb3",
"vnd.docker.reference.type": "attestation-manifest"
},
"platform": {
"architecture": "unknown",
"os": "unknown"
}
},
This means that ko, when given `--platform=all`, will try to pass in a
GOOS "unknown" and GOARCH "unknown" pair to `go build`, which leads to
the image build errors we saw in [1]. This buggy behavior of ko was
fixed in [4], and this change to update ko pulls in that fix. We ran
go get -u github.com/google/ko
go mod tidy
to create this change.
[1] kubernetes#30410
[2] https://ko.build/features/multi-platform/
[3] https://docs.docker.com/build/attestations/attestation-storage/
[4] ko-build/ko#975
listx
pushed a commit
to listx/test-infra
that referenced
this pull request
Sep 6, 2023
The motivation for this change comes from the strange "unsupported
GOOS/GOARCH pair unknown/unknown" errors we were getting in [1] for the
pull-test-infra-prow-image-build-test job. There, we just bumped the
base `git` image from
gcr.io/k8s-prow/git:v20220215-ddc3ad9
to
gcr.io/k8s-prow/git:v20230817-0485b825c2
but this resulted in the aforementioned error for the pod utility
images (clonerefs, sidecar, etc). Digging further, these images are set
to the "all" platform in /.ko.yaml, and this results in the
hack/prowimagebuilder passing the "--platform=all" flag to ko, which
does the actual Docker image build. This instructs ko to look at the
Docker manifest of the base image (in this case the
`gcr.io/k8s-prow/git:...` image), and look through all of the
architectures there, and passes these as is to golang for the go build
invocation [2], which states
To build and push an image for all platforms supported by the
configured base image, simply add --platform=all. This will instruct
ko to look up all the supported platforms in the base image, execute
GOOS=<os> GOARCH=<arch> GOARM=<variant> go build for each platform,
and produce a manifest list containing an image for each platform.
Previously (at least up to gcr.io/k8s-prow/git:v20230111-cd1b3caf9c),
the above behavior worked fine, because the `<os>` and `<arch>` bits in the
Docker manifest for these images looked normal (e.g., "linux", "amd64").
For example, gcr.io/k8s-prow/git:v20220215-ddc3ad9 only has entries that
look like
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:232320cd437e5171fa7e29738e9efa191f714da1ae47d96c1f3b7e3016d15e52",
"size": 1363,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
and also for other architectures like "arm64".
But since gcr.io/k8s-prow/git:v20230324-76cde35b3d the git base image
started having additional "attestation-manifest" annotations [3] in the
Docker manifest that look like
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:5fd1fcaec895346cef283fdf47cf3b061ce7fb4cc8554151eb7b7052ab9886b6",
"size": 566,
"annotations": {
"vnd.docker.reference.digest": "sha256:707be29a6b0a747163add36cb5a594580b4b9dcc7d55dd9324128f8a83d74eb3",
"vnd.docker.reference.type": "attestation-manifest"
},
"platform": {
"architecture": "unknown",
"os": "unknown"
}
},
This means that ko, when given `--platform=all`, will try to pass in a
GOOS "unknown" and GOARCH "unknown" pair to `go build`, which leads to
the image build errors we saw in [1]. This buggy behavior of ko was
fixed in [4], and this change to update ko pulls in that fix.
We ran
hack/make-rules/update/go-deps.sh --minor --tools github.com/google/ko@latest
to create this change.
[1] kubernetes#30410
[2] https://ko.build/features/multi-platform/
[3] https://docs.docker.com/build/attestations/attestation-storage/
[4] ko-build/ko#975
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docker decided to include provenance attestations in index manifests, with descriptors having the platform
unknown/unknown: https://docs.docker.com/build/attestations/attestation-storage/#image-index-sha25694acc2ca70c40f3f6291681f37ce9c767e3d251ce01c7e4e9b98ccf148c26260This causes problems for
ko build --platform=allsinceunknown/unknownis not a, well, known GOOS/GOARCH, in accordance with the OCI image spec.This change ignores
unknown/unknownwhen trying to match a multi-arch base image's platforms, even when using--platform=all.