Document how to set up a local/private registry for caching

[dhiltgen]

Describe the problem/challenge you have
BuildKit has powerful caching constructs which can be utilized to speed up incremental builds on a multi-node environment. For these to work well, you need a registry running "close" to the builder with high bandwidth and low latency. This should be easily achievable with a local private registry running inside the same namespace where the builders run.

Description of the solution you'd like

Write a doc, likely including kube-yaml files in the tree that shows how developers can easily spin up a local registry instance inside the cluster, with a Service so the builders can access it directly for caching purposes. The open source https://github.com/docker/distribution registry should be sufficient for developer use. The registry should not be exposed externally. The docs can be written to pull the yaml directly from this repo (e.g. kubectl apply -f https://raw.githubusercontent.com/vmware-tanzu/buildkit-cli-for-kubectl/main/examples/local-registry.yaml). Those docs should then show how to pass the right flags to kubectl build ... to leverage that local cache (--cache-*) with a "well known" Service name matching the yaml definition (perhaps something like buildkit-cache)

Design/Architecture Details

TBD

Environment Details:

• kubectl buildkit version (use kubectl buildkit version)

v0.1.0

• Kubernetes version (use kubectl version)

NA

• Where are you running kubernetes (e.g., bare metal, vSphere Tanzu, Cloud Provider xKS, etc.)

NA

• Container Runtime and version (e.g. containerd sudo ctr version or dockerd docker version on one of your kubernetes worker nodes)

NA

Vote on this request

This is an invitation to the community to vote on issues. Use the "smiley face" up to the right of this comment to vote.

• 👍 "This project will be more useful if this feature were added"
• 👎 "This feature will not enhance the project in a meaningful way"