Error while joining already joined nodes

[MiroslavRepka]

What happened:
When I tried to apply the updated manifest for the cluster, with new nodes, the kubeone apply threw an error when trying to join already joined nodes.

What is the expected behavior:
Apply the updated manifest and join only the nodes that need to be joined

How to reproduce the issue:

1. Create a manifest for kubeone
2. Run kubeone apply -m <manifest> -y
3. Update the manifest - add new static nodes
4. Run kubeone apply -m <manifest> -y
5. Wait until the kubeone will join the nodes
6. See the error

Anything else we need to know?
After installing kubeone version 1.2.3, the bug was resolved.

Information about the environment:
KubeOne version (kubeone version): 1.3.0 (not sure which alpha, since kubeone reinstalled)
Operating system: Pop!_OS 21.04
Provider you're deploying cluster on: Hetzner
Operating system you're deploying on: Ubuntu

[kron4eg]

@MiroslavRepka few points would help to triage this issue:

• Please provide manifests, the one before the edit and after
• Please grep verbose log from kubeone apply by using additional -v flag and upload it too

[MiroslavRepka]

Original manifest

apiVersion: kubeone.io/v1beta1
kind: KubeOneCluster
name: cluster

versions:
  kubernetes: 'v1.19.0'

clusterNetwork:
  cni:
    external: {}

cloudProvider:
  none: {}
  external: false

addons:
  enable: true
  path: "addons"

apiEndpoint:
  host: 'x.x.x.x'
  port: 6443

controlPlane:
  hosts:
  - publicAddress: 'x.x.x.x'
    privateAddress: '192.168.2.1'
    sshPrivateKeyFile: './private.pem'

staticWorkers:
  hosts:
  - publicAddress: 'x.x.x.x'
    privateAddress: '192.168.2.2'
    sshPrivateKeyFile: './private.pem'

machineController:
  deploy: false

Updated manifest

apiVersion: kubeone.io/v1beta1
kind: KubeOneCluster
name: cluster

versions:
  kubernetes: 'v1.19.0'

clusterNetwork:
  cni:
    external: {}

cloudProvider:
  none: {}
  external: false

addons:
  enable: true
  path: "addons"

apiEndpoint:
  host: 'x.x.x.x'
  port: 6443

controlPlane:
  hosts:
  - publicAddress: 'x.x.x.x'
    privateAddress: '192.168.2.1'
    sshPrivateKeyFile: './private.pem'

staticWorkers:
  hosts:
  - publicAddress: 'x.x.x.x'
    privateAddress: '192.168.2.2'
    sshPrivateKeyFile: './private.pem'
  - publicAddress: 'x.x.x.x'
    privateAddress: '192.168.2.3'
    sshPrivateKeyFile: './private.pem'

machineController:
  deploy: false

I will provide a full log later today since I do not have version 1.3.0 installed anymore. Here is the error message that was occurring for all nodes that were already joined.

ERRO[11:07:25 CEST] + export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/sbin:/usr/local/bin:/opt/bin
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/sbin:/usr/local/bin:/opt/bin
+ [[ -f /etc/kubernetes/admin.conf ]]
+ sudo kubeadm join --config=./kubeone/cfg/worker_1.yaml
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
        [ERROR Port-10250]: Port 10250 is in use
        [ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

@kron4eg

[kron4eg]

@xmudrii didn't we removed recently some small short-circuit "check&exit"? Seems like this is the case. Kubeone should have checked if node is initialized already and skip kubeadm join in that case.

[randrusiak]

I can confirm I have the same issue which only occurs in 1.3.0 alpha versions. In latest stable version everything works.

[xmudrii]

@MiroslavRepka @randrusiak Thank you for reporting the issue! We have fixed it in #1485 and the fix will be included in the upcoming 1.3.0-rc.0 release.