New addons v1beta3 API (#3248)

* New Addons API in KubeOneCluster Signed-off-by: Artiom Diomin <[email protected]> * Fix the code after types changes Signed-off-by: Artiom Diomin <[email protected]> * Automatically convert KubeOneCluster v1beta2 into v1beta3 Signed-off-by: Artiom Diomin <[email protected]> * Migrate e2e test manifests to v1beta3 Signed-off-by: Artiom Diomin <[email protected]> * Add E2E tests for helm addons Signed-off-by: Artiom Diomin <[email protected]> * Make linter happy Signed-off-by: Artiom Diomin <[email protected]> * Fixes NPE in Addons API Signed-off-by: Artiom Diomin <[email protected]> * Convert config template from string into embeddable file Signed-off-by: Artiom Diomin <[email protected]> * Review changes Signed-off-by: Artiom Diomin <[email protected]> * Modernize yamled * replaced interface{} with any * replaced long ifs with more readable switches Signed-off-by: Artiom Diomin <[email protected]> * new yamled Document.Walk() method Signed-off-by: Artiom Diomin <[email protected]> * Framework to free transform YAML * migrate v1beta2 to v1beta3 * fix cosmetic issues Signed-off-by: Artiom Diomin <[email protected]> * Fix config print -f Signed-off-by: Artiom Diomin <[email protected]> * unittests for MigrateV1beta2V1beta3 Signed-off-by: Artiom Diomin <[email protected]> --------- Signed-off-by: Artiom Diomin <[email protected]>
kubermatic · Aug 12, 2024 · 494d081 · 494d081
1 parent 3c79fe3
commit 494d081
Show file tree

Hide file tree

Showing 49 changed files with 4,343 additions and 1,554 deletions.
diff --git a/.prow/generated.yaml b/.prow/generated.yaml
diff --git a/docs/api_reference/v1beta1.en.md b/docs/api_reference/v1beta1.en.md
diff --git a/docs/api_reference/v1beta3.en.md b/docs/api_reference/v1beta3.en.md
@@ -1,13 +1,14 @@
 +++
 title = "v1beta3 API Reference"
-date = 2024-08-02T11:33:54+02:00
+date = 2024-08-05T19:41:51+03:00
 weight = 11
 +++
 ## v1beta3
 
 * [APIEndpoint](#apiendpoint)
 * [AWSSpec](#awsspec)
 * [Addon](#addon)
+* [AddonRef](#addonref)
 * [Addons](#addons)
 * [AzureSpec](#azurespec)
 * [CNI](#cni)
@@ -107,16 +108,25 @@ Addon config
 
 [Back to Group](#v1beta3)
 
+### AddonRef
+
+
+
+| Field | Description | Scheme | Required |
+| ----- | ----------- | ------ | -------- |
+| addon | KubeOne's internal Addon | *[Addon](#addon) | false |
+| helmRelease | HelmReleases configure helm charts to reconcile. For each HelmRelease it will run analog of: `helm upgrade --namespace <NAMESPACE> --install --create-namespace <RELEASE> <CHART> [--values=values-override.yaml]` | *[HelmRelease](#helmrelease) | false |
+
+[Back to Group](#v1beta3)
+
 ### Addons
 
 Addons config
 
 | Field | Description | Scheme | Required |
 | ----- | ----------- | ------ | -------- |
-| enable | Enable | bool | false |
 | path | Path on the local file system to the directory with addons manifests. | string | false |
-| globalParams | GlobalParams to the addon, to render all addons using text/template | map[string]string | false |
-| addons | Addons is a list of config options for named addon | [][Addon](#addon) | false |
+| addons | Addons is a list of config options for named addon | [][AddonRef](#addonref) | false |
 
 [Back to Group](#v1beta3)
 
@@ -420,7 +430,7 @@ GCESpec defines the GCE cloud provider
 | releaseName | ReleaseName is [RELEASE] part of the `helm upgrade [RELEASE] [CHART]` command. Empty is defaulted to chart. | string | false |
 | namespace | Namespace is --namespace flag of the `helm upgrade` command. A namespace to use for a release. | string | true |
 | wait | Wait is --wait flag of the `helm install` command. | bool | false |
-| timeout | WaitTimeout --timeout flag of the `helm install` command. | metav1.Duration | false |
+| timeout | WaitTimeout --timeout flag of the `helm install` command. Default to 5m | metav1.Duration | false |
 | values | Values provide optional overrides of the helm values. | [][HelmValues](#helmvalues) | false |
 
 [Back to Group](#v1beta3)
@@ -519,7 +529,6 @@ KubeOneCluster is KubeOne Cluster API Schema
 | caBundle | CABundle PEM encoded global CA | string | false |
 | features | Features enables and configures additional cluster features. | [Features](#features) | false |
 | addons | Addons are used to deploy additional manifests. | *[Addons](#addons) | false |
-| helmReleases | HelmReleases configure helm charts to reconcile. For each HelmRelease it will run analog of: `helm upgrade --namespace <NAMESPACE> --install --create-namespace <RELEASE> <CHART> [--values=values-override.yaml]` | [][HelmRelease](#helmrelease) | false |
 | systemPackages | SystemPackages configure kubeone behaviour regarding OS packages. | *[SystemPackages](#systempackages) | false |
 | registryConfiguration | RegistryConfiguration configures how Docker images are pulled from an image registry | *[RegistryConfiguration](#registryconfiguration) | false |
 | loggingConfig | LoggingConfig configures the Kubelet's log rotation | [LoggingConfig](#loggingconfig) | false |

diff --git a/pkg/addons/applier.go b/pkg/addons/applier.go
@@ -160,13 +160,6 @@ func newAddonsApplier(s *state.State) (*applier, error) {
 		return nil, err
 	}
 
-	params := map[string]string{}
-	if s.Cluster.Addons.Enabled() {
-		for k, v := range s.Cluster.Addons.GlobalParams {
-			params[k] = v
-		}
-	}
-
 	credsCCM, err := credentials.ProviderCredentials(s.Cluster.CloudProvider, s.CredentialsFilePath, credentials.TypeCCM)
 	if err != nil {
 		return nil, err
@@ -215,7 +208,7 @@ func newAddonsApplier(s *state.State) (*applier, error) {
 			resolver:   s.Images.Get,
 		},
 		Resources: resources.All(),
-		Params:    params,
+		Params:    map[string]string{},
 	}
 
 	if err := csiWebhookCerts(s, &data, csiMigration, kubeCAPrivateKey, kubeCACert); err != nil {

diff --git a/pkg/addons/ensure.go b/pkg/addons/ensure.go
@@ -202,7 +202,7 @@ func EnsureUserAddons(s *state.State) error {
 		}
 	}
 
-	for _, embeddedAddon := range s.Cluster.Addons.Addons {
+	for _, embeddedAddon := range s.Cluster.Addons.OnlyAddons() {
 		if _, ok := embeddedAddons[embeddedAddon.Name]; ok {
 			continue
 		}

diff --git a/pkg/addons/helpers.go b/pkg/addons/helpers.go
@@ -320,15 +320,15 @@ func migrateDaemonsetIfPodSelectorDifferent(s *state.State, key client.ObjectKey
 }
 
 // EmbeddedAddonsOnly checks if all specified addons are embedded addons
-func EmbeddedAddonsOnly(addons []kubeoneapi.Addon) (bool, error) {
+func EmbeddedAddonsOnly(addons *kubeoneapi.Addons) (bool, error) {
 	// Read the directory entries for embedded addons
 	embeddedAddons, err := fs.ReadDir(embeddedaddons.FS, ".")
 	if err != nil {
 		return false, fail.Runtime(err, "reading embedded addons directory")
 	}
 
 	// Iterate over addons specified in the KubeOneCluster object
-	for _, addon := range addons {
+	for _, addon := range addons.OnlyAddons() {
 		embedded := false
 		// Iterate over embedded addons directory to check if the addon exists
 		for _, embeddedAddon := range embeddedAddons {

diff --git a/pkg/addons/list.go b/pkg/addons/list.go
@@ -107,7 +107,7 @@ func List(s *state.State, outputFormat string) error {
 			}
 		}
 
-		for _, embeddedAddon := range s.Cluster.Addons.Addons {
+		for _, embeddedAddon := range s.Cluster.Addons.OnlyAddons() {
 			combinedAddons[embeddedAddon.Name] = addonItem{
 				Name:   embeddedAddon.Name,
 				Status: addonStatusInstall,

diff --git a/pkg/addons/manifest.go b/pkg/addons/manifest.go
@@ -63,7 +63,7 @@ func (a *applier) getManifestsFromDirectory(s *state.State, fsys fs.FS, addonNam
 	}
 
 	if s.Cluster.Addons.Enabled() {
-		for _, addon := range s.Cluster.Addons.Addons {
+		for _, addon := range s.Cluster.Addons.OnlyAddons() {
 			if addon.Name == addonName {
 				addonParams = addon.Params
 				disableTemplating = addon.DisableTemplating

diff --git a/pkg/apis/kubeone/config/migrate.go b/pkg/apis/kubeone/config/migrate.go
@@ -17,45 +17,115 @@ limitations under the License.
 package config
 
 import (
-	"fmt"
+	"bytes"
 	"os"
+	"reflect"
 
+	"gopkg.in/yaml.v2"
+
+	kubeoneapi "k8c.io/kubeone/pkg/apis/kubeone"
+	"k8c.io/kubeone/pkg/apis/kubeone/scheme"
 	kubeonev1beta2 "k8c.io/kubeone/pkg/apis/kubeone/v1beta2"
 	kubeonev1beta3 "k8c.io/kubeone/pkg/apis/kubeone/v1beta3"
 	"k8c.io/kubeone/pkg/fail"
 	"k8c.io/kubeone/pkg/yamled"
+
+	kyaml "sigs.k8s.io/yaml"
 )
 
-// MigrateOldConfig migrates KubeOneCluster v1beta1 object to v1beta2
-func MigrateOldConfig(clusterFilePath string) (interface{}, error) {
-	oldConfig, err := loadClusterConfig(clusterFilePath)
+// MigrateV1beta2V1beta3 migrates KubeOneCluster v1beta2 object to v1beta3
+func MigrateV1beta2V1beta3(clusterFilePath string) ([]byte, error) {
+	originalManifest, err := loadClusterConfig(clusterFilePath)
 	if err != nil {
 		return nil, fail.Runtime(err, "loading cluster config to migrate")
 	}
 
-	// Check is kubeone.k8c.io/v1beta2 config provided
-	apiVersion, apiVersionExists := oldConfig.GetString(yamled.Path{"apiVersion"})
-	if !apiVersionExists {
-		return nil, fail.Config(fmt.Errorf("apiVersion not present in the manifest"), "checking apiVersion presence")
+	var (
+		buffer           bytes.Buffer
+		oldManifest      = kubeonev1beta2.NewKubeOneCluster()
+		newManifest      = kubeonev1beta3.NewKubeOneCluster()
+		internalManifest = new(kubeoneapi.KubeOneCluster)
+	)
+
+	if err = yaml.NewEncoder(&buffer).Encode(originalManifest.Root()); err != nil {
+		return nil, fail.Config(err, "marshaling v1beta2 KubeOneCluster")
+	}
+
+	if err = kyaml.UnmarshalStrict(buffer.Bytes(), oldManifest); err != nil {
+		return nil, fail.Runtime(err, "testing unmarshal v1beta2 KubeOneCluster")
 	}
 
-	if apiVersion != kubeonev1beta2.SchemeGroupVersion.String() {
-		return nil, fail.Config(fmt.Errorf("migration is available only for %q API, but %q is given", kubeonev1beta2.SchemeGroupVersion.String(), apiVersion), "checking apiVersion compatibility")
+	if err = scheme.Scheme.Convert(oldManifest, internalManifest, kubeoneapi.SchemeGroupVersion); err != nil {
+		return nil, fail.Config(err, "converting v1beta2/KubeOneCluster into internal KubeOneCluster")
 	}
 
-	// Ensure kind is KubeOneCluster
-	kind, kindExists := oldConfig.GetString(yamled.Path{"kind"})
-	if !kindExists {
-		return nil, fail.ConfigValidation(fmt.Errorf("kind not present in the manifest"))
+	if err = scheme.Scheme.Convert(internalManifest, newManifest, kubeonev1beta3.SchemeGroupVersion); err != nil {
+		return nil, fail.Config(err, "converting internal KubeOneCluster into v1beta3/KubeOneCluster")
 	}
-	if kind != KubeOneClusterKind {
-		return nil, fail.ConfigValidation(fmt.Errorf("migration is available only for kind %q, but %q is given", KubeOneClusterKind, kind))
+
+	conversionsTab := []struct {
+		path      yamled.Path
+		convertor func(yamled.Path)
+	}{
+		{
+			path: yamled.Path{"apiVersion"},
+			convertor: func(p yamled.Path) {
+				originalManifest.Set(p, kubeonev1beta3.SchemeGroupVersion.String())
+			},
+		},
+		{
+			path: yamled.Path{"addons"},
+			convertor: func(p yamled.Path) {
+				// we moved helmReleases inside the addons
+				if originalManifest.Has(p) || originalManifest.Has(yamled.Path{"helmReleases"}) {
+					ybuf, _ := kyaml.Marshal(newManifest.Addons)
+					addons, _ := yamled.Load(bytes.NewBuffer(ybuf))
+					originalManifest.Set(p, addons)
+				}
+			},
+		},
+		{
+			path: yamled.Path{"addons"},
+			convertor: func(p yamled.Path) {
+				// cleanup addons from all the nil/zero/invalid values
+				originalManifest.Walk(p, func(key yamled.Path, value any) {
+					refval := reflect.ValueOf(value)
+
+					//nolint:exhaustive
+					switch refval.Kind() {
+					case reflect.Pointer, reflect.Map, reflect.Slice:
+						if refval.IsNil() {
+							originalManifest.Remove(key)
+						}
+					case reflect.Invalid:
+						originalManifest.Remove(key)
+					default:
+						if refval.IsZero() {
+							originalManifest.Remove(key)
+						}
+					}
+				})
+			},
+		},
+		{
+			path: yamled.Path{"helmReleases"},
+			convertor: func(p yamled.Path) {
+				originalManifest.Remove(p)
+			},
+		},
 	}
 
-	// The APIVersion has been changed to kubeone.k8c.io/v1beta3
-	oldConfig.Set(yamled.Path{"apiVersion"}, kubeonev1beta3.SchemeGroupVersion.String())
+	for _, conv := range conversionsTab {
+		conv.convertor(conv.path)
+		var buf bytes.Buffer
+		_ = yaml.NewEncoder(&buf).Encode(originalManifest)
+		originalManifest, err = yamled.Load(&buf)
+		if err != nil {
+			return nil, err
+		}
+	}
 
-	return oldConfig.Root(), nil
+	return yaml.Marshal(originalManifest)
 }
 
 // loadClusterConfig takes path to the Cluster Config (old API) and returns yamled.Document

diff --git a/pkg/apis/kubeone/config/migrate_test.go b/pkg/apis/kubeone/config/migrate_test.go
@@ -0,0 +1,46 @@
+/*
+Copyright 2020 The KubeOne Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package config
+
+import (
+	"flag"
+	"testing"
+
+	"k8c.io/kubeone/pkg/testhelper"
+)
+
+var updateFlag = flag.Bool("update", false, "update testdata files")
+
+func TestV1Beta2ToV1Beta3Migration(t *testing.T) {
+	tests := []string{
+		"simple",
+		"just addons",
+		"helm",
+		"addons and helm",
+	}
+
+	for _, test := range tests {
+		t.Run(test, func(t *testing.T) {
+			got, err := MigrateV1beta2V1beta3(testhelper.TestDataFSName(t, "_v1beta2.yaml"))
+			if err != nil {
+				t.Fatalf("%s", err)
+			}
+
+			testhelper.DiffOutput(t, testhelper.FSGoldenName(t), string(got), *updateFlag)
+		})
+	}
+}
diff --git a/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-addons_and_helm.golden b/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-addons_and_helm.golden
@@ -0,0 +1,20 @@
+apiVersion: kubeone.k8c.io/v1beta3
+kind: KubeOneCluster
+versions:
+  kubernetes: 1.30.1
+addons:
+  addons:
+  - addon:
+      name: name1
+  - helmRelease:
+      chart: kube-state-metrics
+      namespace: kube-state-metrics
+      releaseName: ksm
+      repoURL: https://prometheus-community.github.io/helm-charts
+      timeout: 0s
+      values:
+      - valuesFile: ksm-values.yaml
+      - inline:
+          replicas: 3
+      version: 4.22.3
+  path: something
diff --git a/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-addons_and_helm_v1beta2.yaml b/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-addons_and_helm_v1beta2.yaml
@@ -0,0 +1,22 @@
+apiVersion: kubeone.k8c.io/v1beta2
+kind: KubeOneCluster
+
+versions:
+  kubernetes: 1.30.1
+
+addons:
+  path: "something"
+
+  addons:
+    - name: "name1"
+
+helmReleases:
+  - releaseName: ksm
+    chart: kube-state-metrics
+    repoURL: https://prometheus-community.github.io/helm-charts
+    namespace: kube-state-metrics
+    version: 4.22.3
+    values:
+      - valuesFile: ksm-values.yaml
+      - inline:
+          replicas: 3
diff --git a/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-helm.golden b/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-helm.golden
@@ -0,0 +1,17 @@
+apiVersion: kubeone.k8c.io/v1beta3
+kind: KubeOneCluster
+versions:
+  kubernetes: 1.30.1
+addons:
+  addons:
+  - helmRelease:
+      chart: kube-state-metrics
+      namespace: kube-state-metrics
+      releaseName: ksm
+      repoURL: https://prometheus-community.github.io/helm-charts
+      timeout: 0s
+      values:
+      - valuesFile: ksm-values.yaml
+      - inline:
+          replicas: 3
+      version: 4.22.3
diff --git a/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-helm_v1beta2.yaml b/pkg/apis/kubeone/config/testdata/TestV1Beta2ToV1Beta3Migration-helm_v1beta2.yaml
@@ -0,0 +1,16 @@
+apiVersion: kubeone.k8c.io/v1beta2
+kind: KubeOneCluster
+
+versions:
+  kubernetes: 1.30.1
+
+helmReleases:
+  - releaseName: ksm
+    chart: kube-state-metrics
+    repoURL: https://prometheus-community.github.io/helm-charts
+    namespace: kube-state-metrics
+    version: 4.22.3
+    values:
+      - valuesFile: ksm-values.yaml
+      - inline:
+          replicas: 3