Start server with predetermined kubeconfig

[toonsevrin]

Hi, my company kubescale is working on a k3s derivative product (cloud k3s control planes).

One of the small challenges we are facing is the kubeconfig:

1. Is it possible to start a server while you provide the ca data (so we can generate the kubeconfig without accessing the server's file system). Is it sufficient to inject /var/lib/rancher/k3s/server/tls/client-admin.crt and /var/lib/rancher/k3s/server/tls/client-admin.key or will these get overwritten during bootstrapping? If it makes a difference: We run k3s in the HA configuration with a postgresql backend.

2. Perhaps more important for everyone: Is it possible to rotate the kubeconfig?

[brandond]

Are you asking about adding a new server to an existing cluster, or creating a new cluster entirely? Are you talking about a k3s server (apiserver), or agent (kubelet)? You might read through the thread here: #1868 (comment)

[toonsevrin]

@brandond thanks for the response! That issue you refer is extremely useful.

I'm currently talking exclusively about the user facing kubeconfig (admin if I'm not mistaken?).

Because I plan to run k3s in FROM scratch containers (actually currently I just use the rancher/k3s image), getting the kubeconfig out is a bit though. That's why it would be nice to simply be able to populate it ourselves (by mounting in a secret). As discussed in #1868 that should be possible.

The second part of the issue is that I want to be able to give users the option to "regenerate" their kubeconfig. But perhaps it's better practice to not hand-out the admin kubeconfig whatshowever, and create a second kubeconfig from it, that is actually distributed to the user? What do you think?

To give some context regarding the relevance: Kubescale is developing a platform that will manage the k3s control plane for you, and users type "kubescale up" and their current node is networked (wireguard behind the scenes) and connected as a worker to the cluster. So basically we provide the control plane, you provide the workers 😆

[brandond]

You're getting the cart before the horse. If you have access to the CA certs that Kubernetes uses, you can generate as many admin client certs as you want. Just generate the ca certs ahead of time and mount them into your container. You can then use the default admin rbac, or load additional roles and generate client certs to match.