Starting minikube with a custom kube-apiserver image

[yuvalavra]

I would like to test a custom kube-apiserver with minikube, is there a way to configure minikube to use my image instead of the default one (k8s.gcr.io/kube-apiserver-amd64:$version)?

Thanks

[medyagh]

@yuvalavra there is a way to provide your own image registry to minikube, but minikube will look for same image and tag that it expects for each kuberentes version, but if you have your own custom image with same name:tag minikube will use that

      --image-mirror-country='': Country code of the image mirror to be used. Leave empty to use the global one. For Chinese mainland users, set it to cn.
      --image-repository='': Alternative image repository to pull docker images from. This can be used when you have limited access to gcr.io. Set it to "auto" to let minikube decide one for you. For Chinese mainland users, you may use local gcr.io mirrors such as registry.cn-hangzhou.aliyuncs.com/google_containers
      --preload=true: If set, download tarball of preloaded images if available to improve start time. Defaults to true.

[prasadkatti]

So basically you will have to use a custom image repository. You will need to upload all the official images there. Then tag your custom apiserver image to have the same name:tag as the official image for the k8s version you are using and upload that. minikube will then use your custom apiserver image.

[yuvalavra]

Got it, thanks

[prasadkatti]

@yuvalavra - Please let us know if the solution proposed above works for you.

[prasadkatti]

There was an issue (#4104) filed to pull images using tag@hash which was never fixed. That would have made the above workaround unusable. Just an FYI.

[yuvalavra]

Thanks @prasadkatti. I eventually opted to replace the apiserver on the run through the kubelet's manifest directory.

[medyagh]

@yuvalavra do you mind sharing what you did this might be helpful to other users ? or potentially we can add it to our advanced tutorials https://minikube.sigs.k8s.io/docs/tutorials/

[yuvalavra]

@medyagh Gladly, but do know it's a bit hacky and I have no idea whether it'll work for all minikube/k8s versions.

1. Start minikube as usual
2. In the VM, change the image field in the apiserver yaml file to your custom apiserver image (the yaml file should reside in the kubelet's static pods directory at /etc/kubernetes/manifests/kube-apiserver.yaml)
3. Rename the apiserver yaml file to kube-apiserver2.yaml (just needs to be a different name).
4. The kubelet should now notice the old yaml doesn't exist and a new one was created, it'll remove the old apiserver pod and create a new one according to the new yaml file (which now specifies your custom image)

This will only work if the minikube version deploys the apiserver as a static pod, I don't know if all versions do.